2026-02-12T18:07:33.124 INFO:root:teuthology version: 1.2.4.dev5+g8bec0da71 2026-02-12T18:07:33.138 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-12T18:07:33.216 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/yuriw-2026-02-12_15:38:14-rados-wip-yuri4-testing-2026-02-11-2126-distro-default-trial/46524 branch: wip-yuri4-testing-2026-02-11-2126 description: rados/standalone/{supported-random-distro$/{centos_latest} workloads/erasure-code} email: yweinste@redhat.com first_in_suite: false flavor: default job_id: '46524' kernel: branch: distro kdb: true sha1: distro last_in_suite: false machine_type: trial name: yuriw-2026-02-12_15:38:14-rados-wip-yuri4-testing-2026-02-11-2126-distro-default-trial no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-yuri4-testing-2026-02-11-2126 ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 workunit: branch: wip-yuri4-testing-2026-02-11-2126 sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 owner: scheduled_yuriw@soko04.front.sepia.ceph.com priority: 69 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 7172 sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri4-testing-2026-02-11-2126 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_afefaa40503a4bf024e96f97dd7f852b75f00ad6/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 targets: trial116.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBnWBn4+8HjcGWCe6eqqzjrjRXX0jSs8oIJZRSSbK/cH5VRz1Hk776Tls6dfK2R2CGKKWx5mnqnDHt5S9kc9+Zk= tasks: - install: null - workunit: basedir: qa/standalone clients: all: - erasure-code teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 8bec0da71becad44414c54979f64c9ef0e7099c6 timestamp: 2026-02-12_15:38:14 tube: trial user: yuriw verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3932482 2026-02-12T18:07:33.216 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_afefaa40503a4bf024e96f97dd7f852b75f00ad6/qa; will attempt to use it 2026-02-12T18:07:33.217 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_afefaa40503a4bf024e96f97dd7f852b75f00ad6/qa/tasks 2026-02-12T18:07:33.217 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-12T18:07:33.218 INFO:teuthology.task.internal:Checking packages... 2026-02-12T18:07:33.218 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'afefaa40503a4bf024e96f97dd7f852b75f00ad6' 2026-02-12T18:07:33.218 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-12T18:07:33.218 INFO:teuthology.packaging:ref: None 2026-02-12T18:07:33.219 INFO:teuthology.packaging:tag: None 2026-02-12T18:07:33.219 INFO:teuthology.packaging:branch: wip-yuri4-testing-2026-02-11-2126 2026-02-12T18:07:33.219 INFO:teuthology.packaging:sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 2026-02-12T18:07:33.219 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-yuri4-testing-2026-02-11-2126 2026-02-12T18:07:33.341 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5230.gafefaa40 2026-02-12T18:07:33.342 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-12T18:07:33.353 INFO:teuthology.task.internal:no buildpackages task found 2026-02-12T18:07:33.353 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-12T18:07:33.363 INFO:teuthology.task.internal:Saving configuration 2026-02-12T18:07:33.370 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-12T18:07:33.381 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-12T18:07:33.457 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial116.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-12_15:38:14-rados-wip-yuri4-testing-2026-02-11-2126-distro-default-trial/46524', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-12 18:04:13.482473', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBnWBn4+8HjcGWCe6eqqzjrjRXX0jSs8oIJZRSSbK/cH5VRz1Hk776Tls6dfK2R2CGKKWx5mnqnDHt5S9kc9+Zk='} 2026-02-12T18:07:33.458 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-12T18:07:33.469 INFO:teuthology.task.internal:roles: ubuntu@trial116.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-12T18:07:33.469 INFO:teuthology.run_tasks:Running task console_log... 2026-02-12T18:07:33.550 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7dc6c46d9940>, signals=[15]) 2026-02-12T18:07:33.550 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-12T18:07:33.564 INFO:teuthology.task.internal:Opening connections... 2026-02-12T18:07:33.564 DEBUG:teuthology.task.internal:connecting to ubuntu@trial116.front.sepia.ceph.com 2026-02-12T18:07:33.565 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T18:07:33.626 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-12T18:07:33.638 DEBUG:teuthology.orchestra.run.trial116:> uname -m 2026-02-12T18:07:33.700 INFO:teuthology.orchestra.run.trial116.stdout:x86_64 2026-02-12T18:07:33.701 DEBUG:teuthology.orchestra.run.trial116:> cat /etc/os-release 2026-02-12T18:07:33.756 INFO:teuthology.orchestra.run.trial116.stdout:NAME="CentOS Stream" 2026-02-12T18:07:33.756 INFO:teuthology.orchestra.run.trial116.stdout:VERSION="9" 2026-02-12T18:07:33.756 INFO:teuthology.orchestra.run.trial116.stdout:ID="centos" 2026-02-12T18:07:33.756 INFO:teuthology.orchestra.run.trial116.stdout:ID_LIKE="rhel fedora" 2026-02-12T18:07:33.756 INFO:teuthology.orchestra.run.trial116.stdout:VERSION_ID="9" 2026-02-12T18:07:33.756 INFO:teuthology.orchestra.run.trial116.stdout:PLATFORM_ID="platform:el9" 2026-02-12T18:07:33.756 INFO:teuthology.orchestra.run.trial116.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-12T18:07:33.757 INFO:teuthology.orchestra.run.trial116.stdout:ANSI_COLOR="0;31" 2026-02-12T18:07:33.757 INFO:teuthology.orchestra.run.trial116.stdout:LOGO="fedora-logo-icon" 2026-02-12T18:07:33.757 INFO:teuthology.orchestra.run.trial116.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-12T18:07:33.757 INFO:teuthology.orchestra.run.trial116.stdout:HOME_URL="https://centos.org/" 2026-02-12T18:07:33.757 INFO:teuthology.orchestra.run.trial116.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-12T18:07:33.757 INFO:teuthology.orchestra.run.trial116.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-12T18:07:33.757 INFO:teuthology.orchestra.run.trial116.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-12T18:07:33.757 INFO:teuthology.lock.ops:Updating trial116.front.sepia.ceph.com on lock server 2026-02-12T18:07:33.837 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-12T18:07:33.849 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-12T18:07:33.860 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-12T18:07:33.861 DEBUG:teuthology.orchestra.run.trial116:> test '!' -e /home/ubuntu/cephtest 2026-02-12T18:07:33.876 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-12T18:07:33.887 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-12T18:07:33.887 DEBUG:teuthology.orchestra.run.trial116:> test -z $(ls -A /var/lib/ceph) 2026-02-12T18:07:33.931 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-12T18:07:34.042 INFO:teuthology.run_tasks:Running task kernel... 2026-02-12T18:07:34.061 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-12T18:07:34.061 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-12T18:07:34.061 DEBUG:teuthology.orchestra.run.trial116:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-12T18:07:34.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-12T18:07:34.076 DEBUG:teuthology.orchestra.run.trial116:> uname -r 2026-02-12T18:07:34.132 INFO:teuthology.orchestra.run.trial116.stdout:5.14.0-645.el9.x86_64 2026-02-12T18:07:34.133 INFO:teuthology.task.kernel:Running kernel on trial116: 5.14.0-645.el9.x86_64 2026-02-12T18:07:34.133 DEBUG:teuthology.orchestra.run.trial116:> sudo yum install -y kernel 2026-02-12T18:07:36.719 INFO:teuthology.orchestra.run.trial116.stdout:CentOS Stream 9 - BaseOS 4.0 MB/s | 8.9 MB 00:02 2026-02-12T18:08:03.678 INFO:teuthology.orchestra.run.trial116.stdout:CentOS Stream 9 - AppStream 1.0 MB/s | 27 MB 00:26 2026-02-12T18:08:07.570 INFO:teuthology.orchestra.run.trial116.stdout:CentOS Stream 9 - CRB 5.1 MB/s | 7.8 MB 00:01 2026-02-12T18:08:09.270 INFO:teuthology.orchestra.run.trial116.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 20 kB 00:00 2026-02-12T18:08:09.600 INFO:teuthology.orchestra.run.trial116.stdout:Extra Packages for Enterprise Linux 79 MB/s | 20 MB 00:00 2026-02-12T18:08:12.814 INFO:teuthology.orchestra.run.trial116.stdout:lab-extras 1.1 MB/s | 50 kB 00:00 2026-02-12T18:08:13.890 INFO:teuthology.orchestra.run.trial116.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-12T18:08:13.910 INFO:teuthology.orchestra.run.trial116.stdout:Dependencies resolved. 2026-02-12T18:08:13.912 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:08:13.912 INFO:teuthology.orchestra.run.trial116.stdout: Package Architecture Version Repository Size 2026-02-12T18:08:13.912 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:08:13.912 INFO:teuthology.orchestra.run.trial116.stdout:Installing: 2026-02-12T18:08:13.912 INFO:teuthology.orchestra.run.trial116.stdout: kernel x86_64 5.14.0-677.el9 baseos 768 k 2026-02-12T18:08:13.912 INFO:teuthology.orchestra.run.trial116.stdout:Installing dependencies: 2026-02-12T18:08:13.912 INFO:teuthology.orchestra.run.trial116.stdout: kernel-core x86_64 5.14.0-677.el9 baseos 17 M 2026-02-12T18:08:13.912 INFO:teuthology.orchestra.run.trial116.stdout: kernel-modules x86_64 5.14.0-677.el9 baseos 40 M 2026-02-12T18:08:13.912 INFO:teuthology.orchestra.run.trial116.stdout: kernel-modules-core x86_64 5.14.0-677.el9 baseos 31 M 2026-02-12T18:08:13.912 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:08:13.913 INFO:teuthology.orchestra.run.trial116.stdout:Transaction Summary 2026-02-12T18:08:13.913 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:08:13.913 INFO:teuthology.orchestra.run.trial116.stdout:Install 4 Packages 2026-02-12T18:08:13.913 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:08:13.913 INFO:teuthology.orchestra.run.trial116.stdout:Total download size: 88 M 2026-02-12T18:08:13.913 INFO:teuthology.orchestra.run.trial116.stdout:Installed size: 135 M 2026-02-12T18:08:13.913 INFO:teuthology.orchestra.run.trial116.stdout:Downloading Packages: 2026-02-12T18:08:14.129 INFO:teuthology.orchestra.run.trial116.stdout:(1/4): kernel-5.14.0-677.el9.x86_64.rpm 4.8 MB/s | 768 kB 00:00 2026-02-12T18:08:14.763 INFO:teuthology.orchestra.run.trial116.stdout:(2/4): kernel-core-5.14.0-677.el9.x86_64.rpm 22 MB/s | 17 MB 00:00 2026-02-12T18:08:14.804 INFO:teuthology.orchestra.run.trial116.stdout:(3/4): kernel-modules-core-5.14.0-677.el9.x86_6 46 MB/s | 31 MB 00:00 2026-02-12T18:08:15.047 INFO:teuthology.orchestra.run.trial116.stdout:(4/4): kernel-modules-5.14.0-677.el9.x86_64.rpm 37 MB/s | 40 MB 00:01 2026-02-12T18:08:15.047 INFO:teuthology.orchestra.run.trial116.stdout:-------------------------------------------------------------------------------- 2026-02-12T18:08:15.047 INFO:teuthology.orchestra.run.trial116.stdout:Total 78 MB/s | 88 MB 00:01 2026-02-12T18:08:15.238 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction check 2026-02-12T18:08:15.435 INFO:teuthology.orchestra.run.trial116.stdout:Transaction check succeeded. 2026-02-12T18:08:15.435 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction test 2026-02-12T18:08:15.779 INFO:teuthology.orchestra.run.trial116.stdout:Transaction test succeeded. 2026-02-12T18:08:15.779 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction 2026-02-12T18:08:16.877 INFO:teuthology.orchestra.run.trial116.stdout: Preparing : 1/1 2026-02-12T18:08:17.162 INFO:teuthology.orchestra.run.trial116.stdout: Installing : kernel-modules-core-5.14.0-677.el9.x86_64 1/4 2026-02-12T18:08:17.181 INFO:teuthology.orchestra.run.trial116.stdout: Installing : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-12T18:08:18.943 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-12T18:08:18.972 INFO:teuthology.orchestra.run.trial116.stdout: Installing : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-12T18:08:22.234 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-12T18:08:22.240 INFO:teuthology.orchestra.run.trial116.stdout: Installing : kernel-5.14.0-677.el9.x86_64 4/4 2026-02-12T18:08:25.337 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-12T18:08:55.868 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 4/4 2026-02-12T18:08:55.889 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 4/4 2026-02-12T18:08:55.954 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: kernel-5.14.0-677.el9.x86_64 4/4 2026-02-12T18:08:55.954 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : kernel-5.14.0-677.el9.x86_64 1/4 2026-02-12T18:08:55.954 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-12T18:08:55.955 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-12T18:08:56.006 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-12T18:08:56.006 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:08:56.006 INFO:teuthology.orchestra.run.trial116.stdout:Installed: 2026-02-12T18:08:56.006 INFO:teuthology.orchestra.run.trial116.stdout: kernel-5.14.0-677.el9.x86_64 2026-02-12T18:08:56.006 INFO:teuthology.orchestra.run.trial116.stdout: kernel-core-5.14.0-677.el9.x86_64 2026-02-12T18:08:56.006 INFO:teuthology.orchestra.run.trial116.stdout: kernel-modules-5.14.0-677.el9.x86_64 2026-02-12T18:08:56.007 INFO:teuthology.orchestra.run.trial116.stdout: kernel-modules-core-5.14.0-677.el9.x86_64 2026-02-12T18:08:56.007 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:08:56.007 INFO:teuthology.orchestra.run.trial116.stdout:Complete! 2026-02-12T18:08:56.063 DEBUG:teuthology.orchestra.run.trial116:> rpm -q kernel | sort -rV | head -n 1 2026-02-12T18:08:56.090 INFO:teuthology.orchestra.run.trial116.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-12T18:08:56.091 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-12T18:08:56.091 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-645.el9.x86_64 Expected: 5.14.0-677.el9.x86_64 2026-02-12T18:08:56.091 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-12T18:08:56.091 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-12T18:08:56.092 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-12T18:08:56.092 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial116.front.sepia.ceph.com, path=None, version=distro) 2026-02-12T18:08:56.092 DEBUG:teuthology.orchestra.run.trial116:> rpm -q kernel | sort -rV | head -n 1 2026-02-12T18:08:56.155 INFO:teuthology.orchestra.run.trial116.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-12T18:08:56.156 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-12T18:08:56.156 DEBUG:teuthology.orchestra.run.trial116:> sudo rpm -qi grub2-tools 2026-02-12T18:08:56.224 INFO:teuthology.orchestra.run.trial116.stdout:Name : grub2-tools 2026-02-12T18:08:56.225 INFO:teuthology.orchestra.run.trial116.stdout:Epoch : 1 2026-02-12T18:08:56.225 INFO:teuthology.orchestra.run.trial116.stdout:Version : 2.06 2026-02-12T18:08:56.225 INFO:teuthology.orchestra.run.trial116.stdout:Release : 118.el9 2026-02-12T18:08:56.225 INFO:teuthology.orchestra.run.trial116.stdout:Architecture: x86_64 2026-02-12T18:08:56.225 INFO:teuthology.orchestra.run.trial116.stdout:Install Date: Thu 04 Dec 2025 11:54:21 PM UTC 2026-02-12T18:08:56.225 INFO:teuthology.orchestra.run.trial116.stdout:Group : Unspecified 2026-02-12T18:08:56.225 INFO:teuthology.orchestra.run.trial116.stdout:Size : 8393909 2026-02-12T18:08:56.225 INFO:teuthology.orchestra.run.trial116.stdout:License : GPLv3+ 2026-02-12T18:08:56.225 INFO:teuthology.orchestra.run.trial116.stdout:Signature : RSA/SHA256, Wed 05 Nov 2025 09:13:15 AM UTC, Key ID 05b555b38483c65d 2026-02-12T18:08:56.225 INFO:teuthology.orchestra.run.trial116.stdout:Source RPM : grub2-2.06-118.el9.src.rpm 2026-02-12T18:08:56.226 INFO:teuthology.orchestra.run.trial116.stdout:Build Date : Tue 04 Nov 2025 03:44:14 PM UTC 2026-02-12T18:08:56.226 INFO:teuthology.orchestra.run.trial116.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-12T18:08:56.226 INFO:teuthology.orchestra.run.trial116.stdout:Packager : builder@centos.org 2026-02-12T18:08:56.226 INFO:teuthology.orchestra.run.trial116.stdout:Vendor : CentOS 2026-02-12T18:08:56.226 INFO:teuthology.orchestra.run.trial116.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-12T18:08:56.226 INFO:teuthology.orchestra.run.trial116.stdout:Summary : Support tools for GRUB. 2026-02-12T18:08:56.226 INFO:teuthology.orchestra.run.trial116.stdout:Description : 2026-02-12T18:08:56.226 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:08:56.226 INFO:teuthology.orchestra.run.trial116.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-12T18:08:56.226 INFO:teuthology.orchestra.run.trial116.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-12T18:08:56.226 INFO:teuthology.orchestra.run.trial116.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-12T18:08:56.226 INFO:teuthology.orchestra.run.trial116.stdout:hardware devices. 2026-02-12T18:08:56.227 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:08:56.227 INFO:teuthology.orchestra.run.trial116.stdout:This subpackage provides tools for support of all platforms. 2026-02-12T18:08:56.228 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-12T18:08:56.228 INFO:teuthology.task.kernel:Updating grub on trial116 to boot 5.14.0-677.el9.x86_64 2026-02-12T18:08:56.228 DEBUG:teuthology.orchestra.run.trial116:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-12T18:08:56.293 INFO:teuthology.orchestra.run.trial116.stdout:/boot/loader/entries/2734456005d743dca7f3dc77c35a2d0d-0-rescue.conf 2026-02-12T18:08:56.294 DEBUG:teuthology.orchestra.run.trial116:> sudo command -v grubby && echo yes || echo no 2026-02-12T18:08:56.358 INFO:teuthology.orchestra.run.trial116.stdout:/sbin/grubby 2026-02-12T18:08:56.359 INFO:teuthology.orchestra.run.trial116.stdout:yes 2026-02-12T18:08:56.360 DEBUG:teuthology.orchestra.run.trial116:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-677.el9.x86_64" | head -n 1 2026-02-12T18:08:56.423 INFO:teuthology.orchestra.run.trial116.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64.conf 2026-02-12T18:08:56.424 DEBUG:teuthology.orchestra.run.trial116:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64 2026-02-12T18:08:56.497 DEBUG:teuthology.orchestra.run.trial116:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-12T18:08:56.707 INFO:teuthology.orchestra.run.trial116.stderr:Generating grub configuration file ... 2026-02-12T18:08:56.771 INFO:teuthology.orchestra.run.trial116.stderr:Found linux image: /boot/vmlinuz-5.14.0-677.el9.x86_64 2026-02-12T18:08:56.778 INFO:teuthology.orchestra.run.trial116.stderr:Found initrd image: /boot/initramfs-5.14.0-677.el9.x86_64.img 2026-02-12T18:08:57.029 INFO:teuthology.orchestra.run.trial116.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-12T18:08:57.035 INFO:teuthology.orchestra.run.trial116.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-12T18:08:57.067 INFO:teuthology.orchestra.run.trial116.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-12T18:08:57.073 INFO:teuthology.orchestra.run.trial116.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-12T18:08:57.096 INFO:teuthology.orchestra.run.trial116.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-12T18:08:57.103 INFO:teuthology.orchestra.run.trial116.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-12T18:08:57.186 INFO:teuthology.orchestra.run.trial116.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-12T18:08:57.194 INFO:teuthology.orchestra.run.trial116.stderr:done 2026-02-12T18:08:57.196 DEBUG:teuthology.orchestra.run.trial116:> test -d /sys/firmware/efi 2026-02-12T18:08:57.212 DEBUG:teuthology.orchestra.run.trial116:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-12T18:08:57.277 INFO:teuthology.orchestra.run.trial116.stdout:/boot/efi/EFI/BOOT 2026-02-12T18:08:57.278 INFO:teuthology.orchestra.run.trial116.stdout:/boot/efi/EFI/centos 2026-02-12T18:08:57.279 DEBUG:teuthology.orchestra.run.trial116:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-12T18:08:57.345 INFO:teuthology.orchestra.run.trial116.stdout:no 2026-02-12T18:08:57.345 DEBUG:teuthology.orchestra.run.trial116:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-12T18:08:57.409 INFO:teuthology.orchestra.run.trial116.stdout:yes 2026-02-12T18:08:57.410 DEBUG:teuthology.orchestra.run.trial116:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-12T18:08:57.476 DEBUG:teuthology.orchestra.run.trial116:> sudo shutdown -r now 2026-02-12T18:08:57.552 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2026-02-12T18:09:27.550 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-12T18:09:27.550 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial116.front.sepia.ceph.com' 2026-02-12T18:09:27.550 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T18:10:27.613 DEBUG:teuthology.orchestra.remote:timed out 2026-02-12T18:10:36.615 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial116.front.sepia.ceph.com' 2026-02-12T18:10:36.616 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T18:10:36.701 DEBUG:teuthology.orchestra.run.trial116:> true 2026-02-12T18:10:36.863 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial116.front.sepia.ceph.com' 2026-02-12T18:10:36.864 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-677.el9.x86_64"... 2026-02-12T18:10:36.864 DEBUG:teuthology.orchestra.run.trial116:> uname -r 2026-02-12T18:10:36.919 INFO:teuthology.orchestra.run.trial116.stdout:5.14.0-677.el9.x86_64 2026-02-12T18:10:36.919 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-677.el9.x86_64 vs 5.14.0-677.el9.x86_64 2026-02-12T18:10:36.920 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-12T18:10:36.920 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-12T18:10:37.920 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-12T18:10:37.921 DEBUG:teuthology.orchestra.run.trial116:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-12T18:10:37.950 INFO:teuthology.orchestra.run.trial116.stdout:ttyS1 2026-02-12T18:10:37.976 DEBUG:teuthology.parallel:result is None 2026-02-12T18:10:37.976 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-12T18:10:37.988 INFO:teuthology.task.internal:Creating test directory... 2026-02-12T18:10:37.988 DEBUG:teuthology.orchestra.run.trial116:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-12T18:10:38.033 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-12T18:10:38.049 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-12T18:10:38.063 INFO:teuthology.task.internal:Creating archive directory... 2026-02-12T18:10:38.063 DEBUG:teuthology.orchestra.run.trial116:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-12T18:10:38.097 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-12T18:10:38.110 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-12T18:10:38.110 DEBUG:teuthology.orchestra.run.trial116:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-12T18:10:38.155 INFO:teuthology.orchestra.run.trial116.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:10:38.164 INFO:teuthology.orchestra.run.trial116.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:10:38.165 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-12T18:10:38.179 INFO:teuthology.task.internal:Configuring sudo... 2026-02-12T18:10:38.180 DEBUG:teuthology.orchestra.run.trial116:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-12T18:10:38.229 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-12T18:10:38.245 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-12T18:10:38.246 DEBUG:teuthology.orchestra.run.trial116:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-12T18:10:38.285 DEBUG:teuthology.orchestra.run.trial116:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-12T18:10:38.447 DEBUG:teuthology.orchestra.run.trial116:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-12T18:10:38.472 DEBUG:teuthology.orchestra.run.trial116:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-12T18:10:38.606 DEBUG:teuthology.orchestra.run.trial116:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-12T18:10:38.630 DEBUG:teuthology.orchestra.run.trial116:> set -ex 2026-02-12T18:10:38.630 DEBUG:teuthology.orchestra.run.trial116:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-12T18:10:38.692 DEBUG:teuthology.orchestra.run.trial116:> sudo service rsyslog restart 2026-02-12T18:10:38.760 INFO:teuthology.orchestra.run.trial116.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-12T18:10:38.998 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-12T18:10:39.010 INFO:teuthology.task.internal:Starting timer... 2026-02-12T18:10:39.010 INFO:teuthology.run_tasks:Running task pcp... 2026-02-12T18:10:39.033 INFO:teuthology.run_tasks:Running task selinux... 2026-02-12T18:10:39.053 DEBUG:teuthology.orchestra.run.trial116:> sudo service auditd rotate 2026-02-12T18:10:39.082 INFO:teuthology.orchestra.run.trial116.stdout:Rotating logs: 2026-02-12T18:10:39.084 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-12T18:10:39.085 DEBUG:teuthology.orchestra.run.trial116:> /usr/sbin/getenforce 2026-02-12T18:10:39.141 INFO:teuthology.orchestra.run.trial116.stdout:Permissive 2026-02-12T18:10:39.141 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial116.front.sepia.ceph.com': 'permissive'} 2026-02-12T18:10:39.141 DEBUG:teuthology.orchestra.run.trial116:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-12T18:10:39.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-12T18:10:39.206 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-12T18:10:39.206 DEBUG:teuthology.orchestra.run.trial116:> sudo /usr/sbin/setenforce permissive 2026-02-12T18:10:39.269 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-12T18:10:39.288 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2026-02-12T18:10:39.290 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2026-02-12T18:10:39.401 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-12T18:10:39.408 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-12T18:10:39.409 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial116.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-12T18:11:48.625 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial116.front.sepia.ceph.com')] 2026-02-12T18:11:48.627 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial116.front.sepia.ceph.com' 2026-02-12T18:11:48.628 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T18:11:48.736 DEBUG:teuthology.orchestra.run.trial116:> true 2026-02-12T18:11:48.812 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial116.front.sepia.ceph.com' 2026-02-12T18:11:48.812 INFO:teuthology.run_tasks:Running task clock... 2026-02-12T18:11:48.828 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-12T18:11:48.828 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-12T18:11:48.828 DEBUG:teuthology.orchestra.run.trial116:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-12T18:11:48.878 INFO:teuthology.orchestra.run.trial116.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-12T18:11:48.890 INFO:teuthology.orchestra.run.trial116.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-12T18:11:48.910 INFO:teuthology.orchestra.run.trial116.stderr:sudo: ntpd: command not found 2026-02-12T18:11:48.920 INFO:teuthology.orchestra.run.trial116.stdout:506 Cannot talk to daemon 2026-02-12T18:11:48.932 INFO:teuthology.orchestra.run.trial116.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-12T18:11:48.943 INFO:teuthology.orchestra.run.trial116.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-12T18:11:48.990 INFO:teuthology.orchestra.run.trial116.stderr:bash: line 1: ntpq: command not found 2026-02-12T18:11:48.992 INFO:teuthology.orchestra.run.trial116.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-12T18:11:48.992 INFO:teuthology.orchestra.run.trial116.stdout:=============================================================================== 2026-02-12T18:11:48.993 INFO:teuthology.run_tasks:Running task install... 2026-02-12T18:11:49.006 DEBUG:teuthology.task.install:project ceph 2026-02-12T18:11:49.006 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'afefaa40503a4bf024e96f97dd7f852b75f00ad6'}} 2026-02-12T18:11:49.006 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'afefaa40503a4bf024e96f97dd7f852b75f00ad6'} 2026-02-12T18:11:49.006 INFO:teuthology.task.install:Using flavor: default 2026-02-12T18:11:49.015 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-12T18:11:49.015 INFO:teuthology.task.install:extra packages: [] 2026-02-12T18:11:49.016 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'afefaa40503a4bf024e96f97dd7f852b75f00ad6', 'tag': None, 'wait_for_package': False} 2026-02-12T18:11:49.016 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=afefaa40503a4bf024e96f97dd7f852b75f00ad6 2026-02-12T18:11:49.187 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/centos/9/flavors/default/ 2026-02-12T18:11:49.188 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5230.gafefaa40 2026-02-12T18:11:49.574 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-12T18:11:49.574 DEBUG:teuthology.orchestra.run.trial116:> set -ex 2026-02-12T18:11:49.574 DEBUG:teuthology.orchestra.run.trial116:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-12T18:11:49.601 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-12T18:11:49.601 DEBUG:teuthology.orchestra.run.trial116:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/afefaa40503a4bf024e96f97dd7f852b75f00ad6/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-12T18:11:49.667 DEBUG:teuthology.orchestra.run.trial116:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-12T18:11:49.741 DEBUG:teuthology.orchestra.run.trial116:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-12T18:11:49.804 INFO:teuthology.orchestra.run.trial116.stdout:check_obsoletes = 1 2026-02-12T18:11:49.805 DEBUG:teuthology.orchestra.run.trial116:> sudo yum clean all 2026-02-12T18:11:49.958 INFO:teuthology.orchestra.run.trial116.stdout:41 files removed 2026-02-12T18:11:49.972 DEBUG:teuthology.orchestra.run.trial116:> sudo yum -y install ceph-radosgw 2026-02-12T18:11:50.797 INFO:teuthology.orchestra.run.trial116.stdout:ceph packages for x86_64 135 kB/s | 91 kB 00:00 2026-02-12T18:11:51.338 INFO:teuthology.orchestra.run.trial116.stdout:ceph noarch packages 46 kB/s | 24 kB 00:00 2026-02-12T18:11:51.896 INFO:teuthology.orchestra.run.trial116.stdout:ceph source packages 3.9 kB/s | 2.1 kB 00:00 2026-02-12T18:12:00.846 INFO:teuthology.orchestra.run.trial116.stdout:CentOS Stream 9 - BaseOS 1.0 MB/s | 8.9 MB 00:08 2026-02-12T18:12:01.933 INFO:teuthology.orchestra.run.trial116.stdout:CentOS Stream 9 - AppStream 50 MB/s | 27 MB 00:00 2026-02-12T18:12:04.649 INFO:teuthology.orchestra.run.trial116.stdout:CentOS Stream 9 - CRB 25 MB/s | 7.8 MB 00:00 2026-02-12T18:12:06.502 INFO:teuthology.orchestra.run.trial116.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 20 kB 00:01 2026-02-12T18:12:23.598 INFO:teuthology.orchestra.run.trial116.stdout:Extra Packages for Enterprise Linux 1.2 MB/s | 20 MB 00:17 2026-02-12T18:12:26.845 INFO:teuthology.orchestra.run.trial116.stdout:lab-extras 1.7 MB/s | 50 kB 00:00 2026-02-12T18:12:28.136 INFO:teuthology.orchestra.run.trial116.stdout:Dependencies resolved. 2026-02-12T18:12:28.138 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:12:28.138 INFO:teuthology.orchestra.run.trial116.stdout: Package Arch Version Repository Size 2026-02-12T18:12:28.138 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:12:28.138 INFO:teuthology.orchestra.run.trial116.stdout:Installing: 2026-02-12T18:12:28.138 INFO:teuthology.orchestra.run.trial116.stdout: ceph-radosgw x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 13 M 2026-02-12T18:12:28.138 INFO:teuthology.orchestra.run.trial116.stdout:Upgrading: 2026-02-12T18:12:28.138 INFO:teuthology.orchestra.run.trial116.stdout: librados2 x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 3.5 M 2026-02-12T18:12:28.138 INFO:teuthology.orchestra.run.trial116.stdout: librbd1 x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 2.8 M 2026-02-12T18:12:28.138 INFO:teuthology.orchestra.run.trial116.stdout:Installing dependencies: 2026-02-12T18:12:28.138 INFO:teuthology.orchestra.run.trial116.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-12T18:12:28.138 INFO:teuthology.orchestra.run.trial116.stdout: ceph-base x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 5.2 M 2026-02-12T18:12:28.138 INFO:teuthology.orchestra.run.trial116.stdout: ceph-common x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 24 M 2026-02-12T18:12:28.138 INFO:teuthology.orchestra.run.trial116.stdout: ceph-selinux x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 25 k 2026-02-12T18:12:28.138 INFO:teuthology.orchestra.run.trial116.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-12T18:12:28.138 INFO:teuthology.orchestra.run.trial116.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-12T18:12:28.138 INFO:teuthology.orchestra.run.trial116.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-12T18:12:28.139 INFO:teuthology.orchestra.run.trial116.stdout: libcephfs2 x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 909 k 2026-02-12T18:12:28.139 INFO:teuthology.orchestra.run.trial116.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-12T18:12:28.139 INFO:teuthology.orchestra.run.trial116.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-12T18:12:28.139 INFO:teuthology.orchestra.run.trial116.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-12T18:12:28.139 INFO:teuthology.orchestra.run.trial116.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-12T18:12:28.139 INFO:teuthology.orchestra.run.trial116.stdout: libradosstriper1 x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 248 k 2026-02-12T18:12:28.139 INFO:teuthology.orchestra.run.trial116.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-12T18:12:28.139 INFO:teuthology.orchestra.run.trial116.stdout: librgw2 x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 6.5 M 2026-02-12T18:12:28.139 INFO:teuthology.orchestra.run.trial116.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-12T18:12:28.139 INFO:teuthology.orchestra.run.trial116.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-12T18:12:28.139 INFO:teuthology.orchestra.run.trial116.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-12T18:12:28.139 INFO:teuthology.orchestra.run.trial116.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-12T18:12:28.139 INFO:teuthology.orchestra.run.trial116.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-12T18:12:28.139 INFO:teuthology.orchestra.run.trial116.stdout: python3-ceph-argparse x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 45 k 2026-02-12T18:12:28.139 INFO:teuthology.orchestra.run.trial116.stdout: python3-ceph-common x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 185 k 2026-02-12T18:12:28.139 INFO:teuthology.orchestra.run.trial116.stdout: python3-cephfs x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 203 k 2026-02-12T18:12:28.139 INFO:teuthology.orchestra.run.trial116.stdout: python3-rados x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 324 k 2026-02-12T18:12:28.139 INFO:teuthology.orchestra.run.trial116.stdout: python3-rbd x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 305 k 2026-02-12T18:12:28.139 INFO:teuthology.orchestra.run.trial116.stdout: python3-rgw x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 100 k 2026-02-12T18:12:28.139 INFO:teuthology.orchestra.run.trial116.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-12T18:12:28.139 INFO:teuthology.orchestra.run.trial116.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-12T18:12:28.139 INFO:teuthology.orchestra.run.trial116.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-12T18:12:28.140 INFO:teuthology.orchestra.run.trial116.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-12T18:12:28.140 INFO:teuthology.orchestra.run.trial116.stdout:Installing weak dependencies: 2026-02-12T18:12:28.140 INFO:teuthology.orchestra.run.trial116.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-12T18:12:28.140 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:28.140 INFO:teuthology.orchestra.run.trial116.stdout:Transaction Summary 2026-02-12T18:12:28.140 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:12:28.140 INFO:teuthology.orchestra.run.trial116.stdout:Install 32 Packages 2026-02-12T18:12:28.140 INFO:teuthology.orchestra.run.trial116.stdout:Upgrade 2 Packages 2026-02-12T18:12:28.140 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:28.140 INFO:teuthology.orchestra.run.trial116.stdout:Total download size: 67 M 2026-02-12T18:12:28.140 INFO:teuthology.orchestra.run.trial116.stdout:Downloading Packages: 2026-02-12T18:12:29.097 INFO:teuthology.orchestra.run.trial116.stdout:(1/34): ceph-base-20.3.0-5230.gafefaa40.el9.x86 6.9 MB/s | 5.2 MB 00:00 2026-02-12T18:12:29.141 INFO:teuthology.orchestra.run.trial116.stdout:(2/34): ceph-selinux-20.3.0-5230.gafefaa40.el9. 568 kB/s | 25 kB 00:00 2026-02-12T18:12:29.249 INFO:teuthology.orchestra.run.trial116.stdout:(3/34): libcephfs2-20.3.0-5230.gafefaa40.el9.x8 8.3 MB/s | 909 kB 00:00 2026-02-12T18:12:29.300 INFO:teuthology.orchestra.run.trial116.stdout:(4/34): libradosstriper1-20.3.0-5230.gafefaa40. 4.7 MB/s | 248 kB 00:00 2026-02-12T18:12:29.864 INFO:teuthology.orchestra.run.trial116.stdout:(5/34): librgw2-20.3.0-5230.gafefaa40.el9.x86_6 11 MB/s | 6.5 MB 00:00 2026-02-12T18:12:29.920 INFO:teuthology.orchestra.run.trial116.stdout:(6/34): python3-ceph-argparse-20.3.0-5230.gafef 804 kB/s | 45 kB 00:00 2026-02-12T18:12:29.980 INFO:teuthology.orchestra.run.trial116.stdout:(7/34): python3-ceph-common-20.3.0-5230.gafefaa 3.0 MB/s | 185 kB 00:00 2026-02-12T18:12:30.041 INFO:teuthology.orchestra.run.trial116.stdout:(8/34): python3-cephfs-20.3.0-5230.gafefaa40.el 3.2 MB/s | 203 kB 00:00 2026-02-12T18:12:30.107 INFO:teuthology.orchestra.run.trial116.stdout:(9/34): python3-rados-20.3.0-5230.gafefaa40.el9 4.9 MB/s | 324 kB 00:00 2026-02-12T18:12:30.173 INFO:teuthology.orchestra.run.trial116.stdout:(10/34): python3-rbd-20.3.0-5230.gafefaa40.el9. 4.5 MB/s | 305 kB 00:00 2026-02-12T18:12:30.237 INFO:teuthology.orchestra.run.trial116.stdout:(11/34): python3-rgw-20.3.0-5230.gafefaa40.el9. 1.5 MB/s | 100 kB 00:00 2026-02-12T18:12:30.313 INFO:teuthology.orchestra.run.trial116.stdout:(12/34): ceph-radosgw-20.3.0-5230.gafefaa40.el9 6.7 MB/s | 13 MB 00:01 2026-02-12T18:12:30.434 INFO:teuthology.orchestra.run.trial116.stdout:(13/34): mailcap-2.1.49-5.el9.noarch.rpm 168 kB/s | 33 kB 00:00 2026-02-12T18:12:30.508 INFO:teuthology.orchestra.run.trial116.stdout:(14/34): boost-program-options-1.75.0-13.el9.x8 1.4 MB/s | 104 kB 00:00 2026-02-12T18:12:30.551 INFO:teuthology.orchestra.run.trial116.stdout:(15/34): libnbd-1.20.3-4.el9.x86_64.rpm 3.7 MB/s | 164 kB 00:00 2026-02-12T18:12:30.575 INFO:teuthology.orchestra.run.trial116.stdout:(16/34): libpmemobj-1.12.1-1.el9.x86_64.rpm 6.4 MB/s | 160 kB 00:00 2026-02-12T18:12:30.591 INFO:teuthology.orchestra.run.trial116.stdout:(17/34): pciutils-3.7.0-7.el9.x86_64.rpm 336 kB/s | 93 kB 00:00 2026-02-12T18:12:30.597 INFO:teuthology.orchestra.run.trial116.stdout:(18/34): librabbitmq-0.11.0-7.el9.x86_64.rpm 2.0 MB/s | 45 kB 00:00 2026-02-12T18:12:30.621 INFO:teuthology.orchestra.run.trial116.stdout:(19/34): lttng-ust-2.12.0-6.el9.x86_64.rpm 12 MB/s | 292 kB 00:00 2026-02-12T18:12:30.645 INFO:teuthology.orchestra.run.trial116.stdout:(20/34): qatlib-25.08.0-2.el9.x86_64.rpm 9.9 MB/s | 240 kB 00:00 2026-02-12T18:12:30.667 INFO:teuthology.orchestra.run.trial116.stdout:(21/34): qatlib-service-25.08.0-2.el9.x86_64.rp 1.7 MB/s | 37 kB 00:00 2026-02-12T18:12:30.689 INFO:teuthology.orchestra.run.trial116.stdout:(22/34): qatzip-libs-1.3.1-1.el9.x86_64.rpm 2.9 MB/s | 66 kB 00:00 2026-02-12T18:12:30.716 INFO:teuthology.orchestra.run.trial116.stdout:(23/34): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2026-02-12T18:12:30.734 INFO:teuthology.orchestra.run.trial116.stdout:(24/34): gperftools-libs-2.9.1-3.el9.x86_64.rpm 6.8 MB/s | 308 kB 00:00 2026-02-12T18:12:30.743 INFO:teuthology.orchestra.run.trial116.stdout:(25/34): libarrow-doc-9.0.0-15.el9.noarch.rpm 2.6 MB/s | 25 kB 00:00 2026-02-12T18:12:30.749 INFO:teuthology.orchestra.run.trial116.stdout:(26/34): libarrow-9.0.0-15.el9.x86_64.rpm 135 MB/s | 4.4 MB 00:00 2026-02-12T18:12:30.750 INFO:teuthology.orchestra.run.trial116.stdout:(27/34): liboath-2.6.12-1.el9.x86_64.rpm 8.6 MB/s | 49 kB 00:00 2026-02-12T18:12:30.785 INFO:teuthology.orchestra.run.trial116.stdout:(28/34): ceph-common-20.3.0-5230.gafefaa40.el9. 9.9 MB/s | 24 MB 00:02 2026-02-12T18:12:30.786 INFO:teuthology.orchestra.run.trial116.stdout:(29/34): libunwind-1.6.2-1.el9.x86_64.rpm 1.8 MB/s | 67 kB 00:00 2026-02-12T18:12:30.788 INFO:teuthology.orchestra.run.trial116.stdout:(30/34): parquet-libs-9.0.0-15.el9.x86_64.rpm 22 MB/s | 838 kB 00:00 2026-02-12T18:12:30.790 INFO:teuthology.orchestra.run.trial116.stdout:(31/34): re2-20211101-20.el9.x86_64.rpm 46 MB/s | 191 kB 00:00 2026-02-12T18:12:30.795 INFO:teuthology.orchestra.run.trial116.stdout:(32/34): thrift-0.15.0-4.el9.x86_64.rpm 216 MB/s | 1.6 MB 00:00 2026-02-12T18:12:30.973 INFO:teuthology.orchestra.run.trial116.stdout:(33/34): librbd1-20.3.0-5230.gafefaa40.el9.x86_ 15 MB/s | 2.8 MB 00:00 2026-02-12T18:12:31.135 INFO:teuthology.orchestra.run.trial116.stdout:(34/34): librados2-20.3.0-5230.gafefaa40.el9.x8 10 MB/s | 3.5 MB 00:00 2026-02-12T18:12:31.136 INFO:teuthology.orchestra.run.trial116.stdout:-------------------------------------------------------------------------------- 2026-02-12T18:12:31.137 INFO:teuthology.orchestra.run.trial116.stdout:Total 22 MB/s | 67 MB 00:02 2026-02-12T18:12:31.225 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction check 2026-02-12T18:12:31.243 INFO:teuthology.orchestra.run.trial116.stdout:Transaction check succeeded. 2026-02-12T18:12:31.244 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction test 2026-02-12T18:12:31.504 INFO:teuthology.orchestra.run.trial116.stdout:Transaction test succeeded. 2026-02-12T18:12:31.505 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction 2026-02-12T18:12:31.805 INFO:teuthology.orchestra.run.trial116.stdout: Preparing : 1/1 2026-02-12T18:12:31.958 INFO:teuthology.orchestra.run.trial116.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/36 2026-02-12T18:12:31.963 INFO:teuthology.orchestra.run.trial116.stdout: Upgrading : librados2-2:20.3.0-5230.gafefaa40.el9.x86_64 2/36 2026-02-12T18:12:31.993 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: librados2-2:20.3.0-5230.gafefaa40.el9.x86_64 2/36 2026-02-12T18:12:32.033 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-rados-2:20.3.0-5230.gafefaa40.el9.x86_64 3/36 2026-02-12T18:12:32.034 INFO:teuthology.orchestra.run.trial116.stdout: Installing : libcephfs2-2:20.3.0-5230.gafefaa40.el9.x86_64 4/36 2026-02-12T18:12:32.046 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: libcephfs2-2:20.3.0-5230.gafefaa40.el9.x86_64 4/36 2026-02-12T18:12:32.054 INFO:teuthology.orchestra.run.trial116.stdout: Installing : liboath-2.6.12-1.el9.x86_64 5/36 2026-02-12T18:12:32.057 INFO:teuthology.orchestra.run.trial116.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/36 2026-02-12T18:12:32.062 INFO:teuthology.orchestra.run.trial116.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/36 2026-02-12T18:12:32.066 INFO:teuthology.orchestra.run.trial116.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 8/36 2026-02-12T18:12:32.077 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-ceph-argparse-2:20.3.0-5230.gafefaa40.el9. 9/36 2026-02-12T18:12:32.089 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-cephfs-2:20.3.0-5230.gafefaa40.el9.x86_64 10/36 2026-02-12T18:12:32.090 INFO:teuthology.orchestra.run.trial116.stdout: Installing : libradosstriper1-2:20.3.0-5230.gafefaa40.el9.x86_6 11/36 2026-02-12T18:12:32.113 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5230.gafefaa40.el9.x86_6 11/36 2026-02-12T18:12:32.117 INFO:teuthology.orchestra.run.trial116.stdout: Installing : thrift-0.15.0-4.el9.x86_64 12/36 2026-02-12T18:12:32.147 INFO:teuthology.orchestra.run.trial116.stdout: Installing : re2-1:20211101-20.el9.x86_64 13/36 2026-02-12T18:12:32.153 INFO:teuthology.orchestra.run.trial116.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 14/36 2026-02-12T18:12:32.159 INFO:teuthology.orchestra.run.trial116.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 15/36 2026-02-12T18:12:32.162 INFO:teuthology.orchestra.run.trial116.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 16/36 2026-02-12T18:12:32.166 INFO:teuthology.orchestra.run.trial116.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 17/36 2026-02-12T18:12:32.287 INFO:teuthology.orchestra.run.trial116.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/36 2026-02-12T18:12:32.289 INFO:teuthology.orchestra.run.trial116.stdout: Upgrading : librbd1-2:20.3.0-5230.gafefaa40.el9.x86_64 19/36 2026-02-12T18:12:32.315 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: librbd1-2:20.3.0-5230.gafefaa40.el9.x86_64 19/36 2026-02-12T18:12:32.318 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-rbd-2:20.3.0-5230.gafefaa40.el9.x86_64 20/36 2026-02-12T18:12:32.324 INFO:teuthology.orchestra.run.trial116.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/36 2026-02-12T18:12:32.605 INFO:teuthology.orchestra.run.trial116.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/36 2026-02-12T18:12:32.606 INFO:teuthology.orchestra.run.trial116.stdout: Installing : librgw2-2:20.3.0-5230.gafefaa40.el9.x86_64 23/36 2026-02-12T18:12:32.622 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: librgw2-2:20.3.0-5230.gafefaa40.el9.x86_64 23/36 2026-02-12T18:12:32.626 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-rgw-2:20.3.0-5230.gafefaa40.el9.x86_64 24/36 2026-02-12T18:12:32.630 INFO:teuthology.orchestra.run.trial116.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/36 2026-02-12T18:12:32.633 INFO:teuthology.orchestra.run.trial116.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/36 2026-02-12T18:12:32.633 INFO:teuthology.orchestra.run.trial116.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/36 2026-02-12T18:12:32.651 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/36 2026-02-12T18:12:32.654 INFO:teuthology.orchestra.run.trial116.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/36 2026-02-12T18:12:32.687 INFO:teuthology.orchestra.run.trial116.stdout: Installing : mailcap-2.1.49-5.el9.noarch 29/36 2026-02-12T18:12:32.689 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-ceph-common-2:20.3.0-5230.gafefaa40.el9.x8 30/36 2026-02-12T18:12:33.787 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: ceph-common-2:20.3.0-5230.gafefaa40.el9.x86_64 31/36 2026-02-12T18:12:33.789 INFO:teuthology.orchestra.run.trial116.stdout: Installing : ceph-common-2:20.3.0-5230.gafefaa40.el9.x86_64 31/36 2026-02-12T18:12:34.059 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: ceph-common-2:20.3.0-5230.gafefaa40.el9.x86_64 31/36 2026-02-12T18:12:34.062 INFO:teuthology.orchestra.run.trial116.stdout: Installing : ceph-base-2:20.3.0-5230.gafefaa40.el9.x86_64 32/36 2026-02-12T18:12:34.101 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: ceph-base-2:20.3.0-5230.gafefaa40.el9.x86_64 32/36 2026-02-12T18:12:34.101 INFO:teuthology.orchestra.run.trial116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-12T18:12:34.101 INFO:teuthology.orchestra.run.trial116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-12T18:12:34.101 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:34.103 INFO:teuthology.orchestra.run.trial116.stdout: Installing : ceph-selinux-2:20.3.0-5230.gafefaa40.el9.x86_64 33/36 2026-02-12T18:12:38.684 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5230.gafefaa40.el9.x86_64 33/36 2026-02-12T18:12:38.684 INFO:teuthology.orchestra.run.trial116.stdout:skipping the directory /sys 2026-02-12T18:12:38.684 INFO:teuthology.orchestra.run.trial116.stdout:skipping the directory /proc 2026-02-12T18:12:38.684 INFO:teuthology.orchestra.run.trial116.stdout:skipping the directory /mnt 2026-02-12T18:12:38.684 INFO:teuthology.orchestra.run.trial116.stdout:skipping the directory /var/tmp 2026-02-12T18:12:38.684 INFO:teuthology.orchestra.run.trial116.stdout:skipping the directory /home 2026-02-12T18:12:38.684 INFO:teuthology.orchestra.run.trial116.stdout:skipping the directory /root 2026-02-12T18:12:38.684 INFO:teuthology.orchestra.run.trial116.stdout:skipping the directory /tmp 2026-02-12T18:12:38.685 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:39.261 INFO:teuthology.orchestra.run.trial116.stdout: Installing : ceph-radosgw-2:20.3.0-5230.gafefaa40.el9.x86_64 34/36 2026-02-12T18:12:39.285 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5230.gafefaa40.el9.x86_64 34/36 2026-02-12T18:12:39.285 INFO:teuthology.orchestra.run.trial116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T18:12:39.285 INFO:teuthology.orchestra.run.trial116.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-12T18:12:39.285 INFO:teuthology.orchestra.run.trial116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-12T18:12:39.285 INFO:teuthology.orchestra.run.trial116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-12T18:12:39.286 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:39.287 INFO:teuthology.orchestra.run.trial116.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 35/36 2026-02-12T18:12:39.300 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 35/36 2026-02-12T18:12:39.300 INFO:teuthology.orchestra.run.trial116.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 36/36 2026-02-12T18:12:39.355 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 36/36 2026-02-12T18:12:39.355 INFO:teuthology.orchestra.run.trial116.stdout:Creating group 'qat' with GID 992. 2026-02-12T18:12:39.355 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:39.655 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : ceph-base-2:20.3.0-5230.gafefaa40.el9.x86_64 1/36 2026-02-12T18:12:39.656 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : ceph-common-2:20.3.0-5230.gafefaa40.el9.x86_64 2/36 2026-02-12T18:12:39.656 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : ceph-radosgw-2:20.3.0-5230.gafefaa40.el9.x86_64 3/36 2026-02-12T18:12:39.656 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : ceph-selinux-2:20.3.0-5230.gafefaa40.el9.x86_64 4/36 2026-02-12T18:12:39.656 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : libcephfs2-2:20.3.0-5230.gafefaa40.el9.x86_64 5/36 2026-02-12T18:12:39.656 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : libradosstriper1-2:20.3.0-5230.gafefaa40.el9.x86_6 6/36 2026-02-12T18:12:39.656 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : librgw2-2:20.3.0-5230.gafefaa40.el9.x86_64 7/36 2026-02-12T18:12:39.656 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5230.gafefaa40.el9. 8/36 2026-02-12T18:12:39.656 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-ceph-common-2:20.3.0-5230.gafefaa40.el9.x8 9/36 2026-02-12T18:12:39.656 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-cephfs-2:20.3.0-5230.gafefaa40.el9.x86_64 10/36 2026-02-12T18:12:39.657 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-rados-2:20.3.0-5230.gafefaa40.el9.x86_64 11/36 2026-02-12T18:12:39.657 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-rbd-2:20.3.0-5230.gafefaa40.el9.x86_64 12/36 2026-02-12T18:12:39.657 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-rgw-2:20.3.0-5230.gafefaa40.el9.x86_64 13/36 2026-02-12T18:12:39.657 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/36 2026-02-12T18:12:39.657 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 15/36 2026-02-12T18:12:39.657 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 16/36 2026-02-12T18:12:39.657 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 17/36 2026-02-12T18:12:39.657 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 18/36 2026-02-12T18:12:39.657 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 19/36 2026-02-12T18:12:39.657 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 20/36 2026-02-12T18:12:39.657 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 21/36 2026-02-12T18:12:39.657 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 22/36 2026-02-12T18:12:39.658 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 23/36 2026-02-12T18:12:39.658 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 24/36 2026-02-12T18:12:39.658 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 25/36 2026-02-12T18:12:39.658 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 26/36 2026-02-12T18:12:39.658 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 27/36 2026-02-12T18:12:39.658 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 28/36 2026-02-12T18:12:39.658 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 29/36 2026-02-12T18:12:39.658 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 30/36 2026-02-12T18:12:39.658 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : re2-1:20211101-20.el9.x86_64 31/36 2026-02-12T18:12:39.658 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 32/36 2026-02-12T18:12:39.658 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : librados2-2:20.3.0-5230.gafefaa40.el9.x86_64 33/36 2026-02-12T18:12:39.658 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 34/36 2026-02-12T18:12:39.658 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : librbd1-2:20.3.0-5230.gafefaa40.el9.x86_64 35/36 2026-02-12T18:12:39.716 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 36/36 2026-02-12T18:12:39.717 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:39.717 INFO:teuthology.orchestra.run.trial116.stdout:Upgraded: 2026-02-12T18:12:39.717 INFO:teuthology.orchestra.run.trial116.stdout: librados2-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:12:39.717 INFO:teuthology.orchestra.run.trial116.stdout: librbd1-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:12:39.717 INFO:teuthology.orchestra.run.trial116.stdout:Installed: 2026-02-12T18:12:39.717 INFO:teuthology.orchestra.run.trial116.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-12T18:12:39.717 INFO:teuthology.orchestra.run.trial116.stdout: ceph-base-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:12:39.717 INFO:teuthology.orchestra.run.trial116.stdout: ceph-common-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:12:39.717 INFO:teuthology.orchestra.run.trial116.stdout: ceph-radosgw-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:12:39.717 INFO:teuthology.orchestra.run.trial116.stdout: ceph-selinux-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:12:39.718 INFO:teuthology.orchestra.run.trial116.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-12T18:12:39.718 INFO:teuthology.orchestra.run.trial116.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-12T18:12:39.718 INFO:teuthology.orchestra.run.trial116.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-12T18:12:39.718 INFO:teuthology.orchestra.run.trial116.stdout: libcephfs2-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:12:39.718 INFO:teuthology.orchestra.run.trial116.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-12T18:12:39.718 INFO:teuthology.orchestra.run.trial116.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-12T18:12:39.718 INFO:teuthology.orchestra.run.trial116.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-12T18:12:39.718 INFO:teuthology.orchestra.run.trial116.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-12T18:12:39.718 INFO:teuthology.orchestra.run.trial116.stdout: libradosstriper1-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:12:39.718 INFO:teuthology.orchestra.run.trial116.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-12T18:12:39.718 INFO:teuthology.orchestra.run.trial116.stdout: librgw2-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:12:39.718 INFO:teuthology.orchestra.run.trial116.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-12T18:12:39.718 INFO:teuthology.orchestra.run.trial116.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-12T18:12:39.719 INFO:teuthology.orchestra.run.trial116.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-12T18:12:39.719 INFO:teuthology.orchestra.run.trial116.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-12T18:12:39.719 INFO:teuthology.orchestra.run.trial116.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-12T18:12:39.719 INFO:teuthology.orchestra.run.trial116.stdout: python3-ceph-argparse-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:12:39.719 INFO:teuthology.orchestra.run.trial116.stdout: python3-ceph-common-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:12:39.719 INFO:teuthology.orchestra.run.trial116.stdout: python3-cephfs-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:12:39.719 INFO:teuthology.orchestra.run.trial116.stdout: python3-rados-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:12:39.719 INFO:teuthology.orchestra.run.trial116.stdout: python3-rbd-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:12:39.719 INFO:teuthology.orchestra.run.trial116.stdout: python3-rgw-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:12:39.719 INFO:teuthology.orchestra.run.trial116.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-12T18:12:39.719 INFO:teuthology.orchestra.run.trial116.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-12T18:12:39.719 INFO:teuthology.orchestra.run.trial116.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-12T18:12:39.720 INFO:teuthology.orchestra.run.trial116.stdout: re2-1:20211101-20.el9.x86_64 2026-02-12T18:12:39.720 INFO:teuthology.orchestra.run.trial116.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-12T18:12:39.720 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:39.720 INFO:teuthology.orchestra.run.trial116.stdout:Complete! 2026-02-12T18:12:39.758 DEBUG:teuthology.orchestra.run.trial116:> sudo yum -y install ceph-test 2026-02-12T18:12:40.062 INFO:teuthology.orchestra.run.trial116.stdout:Last metadata expiration check: 0:00:14 ago on Thu 12 Feb 2026 06:12:26 PM UTC. 2026-02-12T18:12:40.146 INFO:teuthology.orchestra.run.trial116.stdout:Dependencies resolved. 2026-02-12T18:12:40.147 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:12:40.147 INFO:teuthology.orchestra.run.trial116.stdout: Package Arch Version Repository Size 2026-02-12T18:12:40.147 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:12:40.147 INFO:teuthology.orchestra.run.trial116.stdout:Installing: 2026-02-12T18:12:40.147 INFO:teuthology.orchestra.run.trial116.stdout: ceph-test x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 78 M 2026-02-12T18:12:40.147 INFO:teuthology.orchestra.run.trial116.stdout:Installing dependencies: 2026-02-12T18:12:40.147 INFO:teuthology.orchestra.run.trial116.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-12T18:12:40.147 INFO:teuthology.orchestra.run.trial116.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-12T18:12:40.148 INFO:teuthology.orchestra.run.trial116.stdout: libcephsqlite x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 159 k 2026-02-12T18:12:40.148 INFO:teuthology.orchestra.run.trial116.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-12T18:12:40.148 INFO:teuthology.orchestra.run.trial116.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-12T18:12:40.148 INFO:teuthology.orchestra.run.trial116.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-12T18:12:40.148 INFO:teuthology.orchestra.run.trial116.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-12T18:12:40.148 INFO:teuthology.orchestra.run.trial116.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-12T18:12:40.148 INFO:teuthology.orchestra.run.trial116.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-12T18:12:40.148 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:40.148 INFO:teuthology.orchestra.run.trial116.stdout:Transaction Summary 2026-02-12T18:12:40.148 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:12:40.148 INFO:teuthology.orchestra.run.trial116.stdout:Install 10 Packages 2026-02-12T18:12:40.148 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:40.149 INFO:teuthology.orchestra.run.trial116.stdout:Total download size: 82 M 2026-02-12T18:12:40.149 INFO:teuthology.orchestra.run.trial116.stdout:Installed size: 339 M 2026-02-12T18:12:40.149 INFO:teuthology.orchestra.run.trial116.stdout:Downloading Packages: 2026-02-12T18:12:40.720 INFO:teuthology.orchestra.run.trial116.stdout:(1/10): libcephsqlite-20.3.0-5230.gafefaa40.el9 1.6 MB/s | 159 kB 00:00 2026-02-12T18:12:41.274 INFO:teuthology.orchestra.run.trial116.stdout:(2/10): libatomic-11.5.0-14.el9.x86_64.rpm 41 kB/s | 23 kB 00:00 2026-02-12T18:12:41.509 INFO:teuthology.orchestra.run.trial116.stdout:(3/10): lksctp-tools-1.0.19-2.el9.x86_64.rpm 400 kB/s | 94 kB 00:00 2026-02-12T18:12:41.901 INFO:teuthology.orchestra.run.trial116.stdout:(4/10): libxslt-1.1.34-12.el9.x86_64.rpm 596 kB/s | 233 kB 00:00 2026-02-12T18:12:42.591 INFO:teuthology.orchestra.run.trial116.stdout:(5/10): ceph-test-20.3.0-5230.gafefaa40.el9.x86 40 MB/s | 78 MB 00:01 2026-02-12T18:12:43.151 INFO:teuthology.orchestra.run.trial116.stdout:(6/10): protobuf-3.14.0-17.el9.x86_64.rpm 823 kB/s | 1.0 MB 00:01 2026-02-12T18:12:43.309 INFO:teuthology.orchestra.run.trial116.stdout:(7/10): xmlstarlet-1.6.1-20.el9.x86_64.rpm 402 kB/s | 64 kB 00:00 2026-02-12T18:12:43.456 INFO:teuthology.orchestra.run.trial116.stdout:(8/10): socat-1.7.4.1-8.el9.x86_64.rpm 350 kB/s | 303 kB 00:00 2026-02-12T18:12:43.588 INFO:teuthology.orchestra.run.trial116.stdout:(9/10): yaml-cpp-0.6.3-5.el9.x86_64.rpm 437 kB/s | 122 kB 00:00 2026-02-12T18:12:43.649 INFO:teuthology.orchestra.run.trial116.stdout:(10/10): hwloc-libs-2.4.1-6.el9.x86_64.rpm 709 kB/s | 2.1 MB 00:03 2026-02-12T18:12:43.653 INFO:teuthology.orchestra.run.trial116.stdout:-------------------------------------------------------------------------------- 2026-02-12T18:12:43.653 INFO:teuthology.orchestra.run.trial116.stdout:Total 23 MB/s | 82 MB 00:03 2026-02-12T18:12:43.714 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction check 2026-02-12T18:12:43.725 INFO:teuthology.orchestra.run.trial116.stdout:Transaction check succeeded. 2026-02-12T18:12:43.725 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction test 2026-02-12T18:12:44.009 INFO:teuthology.orchestra.run.trial116.stdout:Transaction test succeeded. 2026-02-12T18:12:44.009 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction 2026-02-12T18:12:44.317 INFO:teuthology.orchestra.run.trial116.stdout: Preparing : 1/1 2026-02-12T18:12:44.326 INFO:teuthology.orchestra.run.trial116.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/10 2026-02-12T18:12:44.334 INFO:teuthology.orchestra.run.trial116.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/10 2026-02-12T18:12:44.340 INFO:teuthology.orchestra.run.trial116.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/10 2026-02-12T18:12:44.344 INFO:teuthology.orchestra.run.trial116.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/10 2026-02-12T18:12:44.350 INFO:teuthology.orchestra.run.trial116.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/10 2026-02-12T18:12:44.351 INFO:teuthology.orchestra.run.trial116.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 6/10 2026-02-12T18:12:44.361 INFO:teuthology.orchestra.run.trial116.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 7/10 2026-02-12T18:12:44.369 INFO:teuthology.orchestra.run.trial116.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 8/10 2026-02-12T18:12:44.370 INFO:teuthology.orchestra.run.trial116.stdout: Installing : libcephsqlite-2:20.3.0-5230.gafefaa40.el9.x86_64 9/10 2026-02-12T18:12:47.799 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5230.gafefaa40.el9.x86_64 9/10 2026-02-12T18:12:47.818 INFO:teuthology.orchestra.run.trial116.stdout: Installing : ceph-test-2:20.3.0-5230.gafefaa40.el9.x86_64 10/10 2026-02-12T18:12:48.138 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: ceph-test-2:20.3.0-5230.gafefaa40.el9.x86_64 10/10 2026-02-12T18:12:48.139 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : ceph-test-2:20.3.0-5230.gafefaa40.el9.x86_64 1/10 2026-02-12T18:12:48.139 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : libcephsqlite-2:20.3.0-5230.gafefaa40.el9.x86_64 2/10 2026-02-12T18:12:48.139 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/10 2026-02-12T18:12:48.139 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/10 2026-02-12T18:12:48.139 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/10 2026-02-12T18:12:48.139 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 6/10 2026-02-12T18:12:48.139 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 7/10 2026-02-12T18:12:48.139 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 8/10 2026-02-12T18:12:48.139 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 9/10 2026-02-12T18:12:48.194 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 10/10 2026-02-12T18:12:48.194 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:48.194 INFO:teuthology.orchestra.run.trial116.stdout:Installed: 2026-02-12T18:12:48.195 INFO:teuthology.orchestra.run.trial116.stdout: ceph-test-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:12:48.195 INFO:teuthology.orchestra.run.trial116.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-12T18:12:48.195 INFO:teuthology.orchestra.run.trial116.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-12T18:12:48.195 INFO:teuthology.orchestra.run.trial116.stdout: libcephsqlite-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:12:48.195 INFO:teuthology.orchestra.run.trial116.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-12T18:12:48.195 INFO:teuthology.orchestra.run.trial116.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-12T18:12:48.195 INFO:teuthology.orchestra.run.trial116.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-12T18:12:48.195 INFO:teuthology.orchestra.run.trial116.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-12T18:12:48.195 INFO:teuthology.orchestra.run.trial116.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-12T18:12:48.195 INFO:teuthology.orchestra.run.trial116.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-12T18:12:48.195 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:48.195 INFO:teuthology.orchestra.run.trial116.stdout:Complete! 2026-02-12T18:12:48.232 DEBUG:teuthology.orchestra.run.trial116:> sudo yum -y install ceph 2026-02-12T18:12:48.531 INFO:teuthology.orchestra.run.trial116.stdout:Last metadata expiration check: 0:00:22 ago on Thu 12 Feb 2026 06:12:26 PM UTC. 2026-02-12T18:12:48.616 INFO:teuthology.orchestra.run.trial116.stdout:Dependencies resolved. 2026-02-12T18:12:48.617 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:12:48.617 INFO:teuthology.orchestra.run.trial116.stdout: Package Arch Version Repository Size 2026-02-12T18:12:48.617 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:12:48.618 INFO:teuthology.orchestra.run.trial116.stdout:Installing: 2026-02-12T18:12:48.618 INFO:teuthology.orchestra.run.trial116.stdout: ceph x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 6.5 k 2026-02-12T18:12:48.618 INFO:teuthology.orchestra.run.trial116.stdout:Installing dependencies: 2026-02-12T18:12:48.618 INFO:teuthology.orchestra.run.trial116.stdout: ceph-mds x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 2.4 M 2026-02-12T18:12:48.618 INFO:teuthology.orchestra.run.trial116.stdout: ceph-mgr x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 936 k 2026-02-12T18:12:48.618 INFO:teuthology.orchestra.run.trial116.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 297 k 2026-02-12T18:12:48.618 INFO:teuthology.orchestra.run.trial116.stdout: ceph-mon x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 5.0 M 2026-02-12T18:12:48.618 INFO:teuthology.orchestra.run.trial116.stdout: ceph-osd x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 166 k 2026-02-12T18:12:48.618 INFO:teuthology.orchestra.run.trial116.stdout: ceph-osd-classic x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 16 M 2026-02-12T18:12:48.618 INFO:teuthology.orchestra.run.trial116.stdout: ceph-osd-crimson x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 19 M 2026-02-12T18:12:48.618 INFO:teuthology.orchestra.run.trial116.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-12T18:12:48.619 INFO:teuthology.orchestra.run.trial116.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-12T18:12:48.619 INFO:teuthology.orchestra.run.trial116.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-12T18:12:48.619 INFO:teuthology.orchestra.run.trial116.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-12T18:12:48.619 INFO:teuthology.orchestra.run.trial116.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-12T18:12:48.619 INFO:teuthology.orchestra.run.trial116.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-12T18:12:48.619 INFO:teuthology.orchestra.run.trial116.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-12T18:12:48.619 INFO:teuthology.orchestra.run.trial116.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-12T18:12:48.619 INFO:teuthology.orchestra.run.trial116.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-12T18:12:48.619 INFO:teuthology.orchestra.run.trial116.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-12T18:12:48.619 INFO:teuthology.orchestra.run.trial116.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-12T18:12:48.619 INFO:teuthology.orchestra.run.trial116.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-12T18:12:48.619 INFO:teuthology.orchestra.run.trial116.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-12T18:12:48.619 INFO:teuthology.orchestra.run.trial116.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-12T18:12:48.620 INFO:teuthology.orchestra.run.trial116.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-12T18:12:48.620 INFO:teuthology.orchestra.run.trial116.stdout: python3-jaraco-collections 2026-02-12T18:12:48.620 INFO:teuthology.orchestra.run.trial116.stdout: noarch 3.0.0-8.el9 epel 23 k 2026-02-12T18:12:48.620 INFO:teuthology.orchestra.run.trial116.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-12T18:12:48.620 INFO:teuthology.orchestra.run.trial116.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-12T18:12:48.620 INFO:teuthology.orchestra.run.trial116.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-12T18:12:48.620 INFO:teuthology.orchestra.run.trial116.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-12T18:12:48.620 INFO:teuthology.orchestra.run.trial116.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-12T18:12:48.620 INFO:teuthology.orchestra.run.trial116.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-12T18:12:48.620 INFO:teuthology.orchestra.run.trial116.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-12T18:12:48.620 INFO:teuthology.orchestra.run.trial116.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-12T18:12:48.620 INFO:teuthology.orchestra.run.trial116.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-12T18:12:48.621 INFO:teuthology.orchestra.run.trial116.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-12T18:12:48.621 INFO:teuthology.orchestra.run.trial116.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-12T18:12:48.621 INFO:teuthology.orchestra.run.trial116.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-12T18:12:48.621 INFO:teuthology.orchestra.run.trial116.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-12T18:12:48.621 INFO:teuthology.orchestra.run.trial116.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-12T18:12:48.621 INFO:teuthology.orchestra.run.trial116.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-12T18:12:48.621 INFO:teuthology.orchestra.run.trial116.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-12T18:12:48.621 INFO:teuthology.orchestra.run.trial116.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-12T18:12:48.621 INFO:teuthology.orchestra.run.trial116.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-12T18:12:48.621 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:48.621 INFO:teuthology.orchestra.run.trial116.stdout:Transaction Summary 2026-02-12T18:12:48.621 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:12:48.621 INFO:teuthology.orchestra.run.trial116.stdout:Install 42 Packages 2026-02-12T18:12:48.622 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:48.622 INFO:teuthology.orchestra.run.trial116.stdout:Total download size: 48 M 2026-02-12T18:12:48.622 INFO:teuthology.orchestra.run.trial116.stdout:Installed size: 167 M 2026-02-12T18:12:48.623 INFO:teuthology.orchestra.run.trial116.stdout:Downloading Packages: 2026-02-12T18:12:49.386 INFO:teuthology.orchestra.run.trial116.stdout:(1/42): ceph-20.3.0-5230.gafefaa40.el9.x86_64.r 82 kB/s | 6.5 kB 00:00 2026-02-12T18:12:49.569 INFO:teuthology.orchestra.run.trial116.stdout:(2/42): ceph-mgr-20.3.0-5230.gafefaa40.el9.x86_ 3.5 MB/s | 936 kB 00:00 2026-02-12T18:12:49.602 INFO:teuthology.orchestra.run.trial116.stdout:(3/42): ceph-osd-20.3.0-5230.gafefaa40.el9.x86_ 5.0 MB/s | 166 kB 00:00 2026-02-12T18:12:49.622 INFO:teuthology.orchestra.run.trial116.stdout:(4/42): ceph-mds-20.3.0-5230.gafefaa40.el9.x86_ 7.5 MB/s | 2.4 MB 00:00 2026-02-12T18:12:50.028 INFO:teuthology.orchestra.run.trial116.stdout:(5/42): ceph-mon-20.3.0-5230.gafefaa40.el9.x86_ 7.8 MB/s | 5.0 MB 00:00 2026-02-12T18:12:50.075 INFO:teuthology.orchestra.run.trial116.stdout:(6/42): ceph-mgr-modules-core-20.3.0-5230.gafef 6.2 MB/s | 297 kB 00:00 2026-02-12T18:12:50.363 INFO:teuthology.orchestra.run.trial116.stdout:(7/42): ledmon-libs-1.1.0-3.el9.x86_64.rpm 141 kB/s | 40 kB 00:00 2026-02-12T18:12:50.553 INFO:teuthology.orchestra.run.trial116.stdout:(8/42): libconfig-1.7.2-9.el9.x86_64.rpm 378 kB/s | 72 kB 00:00 2026-02-12T18:12:50.592 INFO:teuthology.orchestra.run.trial116.stdout:(9/42): ceph-osd-crimson-20.3.0-5230.gafefaa40. 20 MB/s | 19 MB 00:00 2026-02-12T18:12:50.722 INFO:teuthology.orchestra.run.trial116.stdout:(10/42): ceph-osd-classic-20.3.0-5230.gafefaa40 14 MB/s | 16 MB 00:01 2026-02-12T18:12:50.781 INFO:teuthology.orchestra.run.trial116.stdout:(11/42): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2026-02-12T18:12:50.888 INFO:teuthology.orchestra.run.trial116.stdout:(12/42): python3-pycparser-2.20-6.el9.noarch.rp 1.2 MB/s | 135 kB 00:00 2026-02-12T18:12:50.998 INFO:teuthology.orchestra.run.trial116.stdout:(13/42): python3-pyparsing-2.4.7-9.el9.noarch.r 1.3 MB/s | 150 kB 00:00 2026-02-12T18:12:51.102 INFO:teuthology.orchestra.run.trial116.stdout:(14/42): python3-requests-2.25.1-10.el9.noarch. 1.2 MB/s | 126 kB 00:00 2026-02-12T18:12:51.126 INFO:teuthology.orchestra.run.trial116.stdout:(15/42): python3-ply-3.11-14.el9.noarch.rpm 263 kB/s | 106 kB 00:00 2026-02-12T18:12:51.227 INFO:teuthology.orchestra.run.trial116.stdout:(16/42): python3-urllib3-1.26.5-7.el9.noarch.rp 1.7 MB/s | 218 kB 00:00 2026-02-12T18:12:51.280 INFO:teuthology.orchestra.run.trial116.stdout:(17/42): python3-cryptography-36.0.1-5.el9.x86_ 1.8 MB/s | 1.2 MB 00:00 2026-02-12T18:12:51.313 INFO:teuthology.orchestra.run.trial116.stdout:(18/42): unzip-6.0-59.el9.x86_64.rpm 973 kB/s | 182 kB 00:00 2026-02-12T18:12:51.356 INFO:teuthology.orchestra.run.trial116.stdout:(19/42): zip-3.0-35.el9.x86_64.rpm 2.0 MB/s | 266 kB 00:00 2026-02-12T18:12:51.397 INFO:teuthology.orchestra.run.trial116.stdout:(20/42): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2026-02-12T18:12:51.398 INFO:teuthology.orchestra.run.trial116.stdout:(21/42): python3-libstoragemgmt-1.10.1-1.el9.x8 4.2 MB/s | 177 kB 00:00 2026-02-12T18:12:51.400 INFO:teuthology.orchestra.run.trial116.stdout:(22/42): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 2.0 MB/s | 246 kB 00:00 2026-02-12T18:12:51.403 INFO:teuthology.orchestra.run.trial116.stdout:(23/42): python3-toml-0.10.2-6.el9.noarch.rpm 8.3 MB/s | 42 kB 00:00 2026-02-12T18:12:51.403 INFO:teuthology.orchestra.run.trial116.stdout:(24/42): python3-packaging-20.9-5.el9.noarch.rp 13 MB/s | 77 kB 00:00 2026-02-12T18:12:51.589 INFO:teuthology.orchestra.run.trial116.stdout:(25/42): python3-autocommand-2.2.2-8.el9.noarch 158 kB/s | 29 kB 00:00 2026-02-12T18:12:51.710 INFO:teuthology.orchestra.run.trial116.stdout:(26/42): python3-backports-tarfile-1.2.0-1.el9. 499 kB/s | 60 kB 00:00 2026-02-12T18:12:51.712 INFO:teuthology.orchestra.run.trial116.stdout:(27/42): luarocks-3.9.2-5.el9.noarch.rpm 489 kB/s | 151 kB 00:00 2026-02-12T18:12:51.779 INFO:teuthology.orchestra.run.trial116.stdout:(28/42): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 629 kB/s | 43 kB 00:00 2026-02-12T18:12:51.818 INFO:teuthology.orchestra.run.trial116.stdout:(29/42): python3-cheroot-10.0.1-4.el9.noarch.rp 1.6 MB/s | 173 kB 00:00 2026-02-12T18:12:51.845 INFO:teuthology.orchestra.run.trial116.stdout:(30/42): lua-devel-5.4.4-4.el9.x86_64.rpm 50 kB/s | 22 kB 00:00 2026-02-12T18:12:51.872 INFO:teuthology.orchestra.run.trial116.stdout:(31/42): python3-jaraco-8.2.1-3.el9.noarch.rpm 199 kB/s | 11 kB 00:00 2026-02-12T18:12:51.926 INFO:teuthology.orchestra.run.trial116.stdout:(32/42): python3-jaraco-collections-3.0.0-8.el9 429 kB/s | 23 kB 00:00 2026-02-12T18:12:51.947 INFO:teuthology.orchestra.run.trial116.stdout:(33/42): python3-cherrypy-18.6.1-2.el9.noarch.r 2.1 MB/s | 358 kB 00:00 2026-02-12T18:12:51.980 INFO:teuthology.orchestra.run.trial116.stdout:(34/42): python3-jaraco-context-6.0.1-3.el9.noa 364 kB/s | 20 kB 00:00 2026-02-12T18:12:51.999 INFO:teuthology.orchestra.run.trial116.stdout:(35/42): python3-jaraco-classes-3.2.1-5.el9.noa 115 kB/s | 18 kB 00:00 2026-02-12T18:12:52.000 INFO:teuthology.orchestra.run.trial116.stdout:(36/42): python3-jaraco-functools-3.5.0-2.el9.n 362 kB/s | 19 kB 00:00 2026-02-12T18:12:52.035 INFO:teuthology.orchestra.run.trial116.stdout:(37/42): python3-jaraco-text-4.0.0-2.el9.noarch 479 kB/s | 26 kB 00:00 2026-02-12T18:12:52.056 INFO:teuthology.orchestra.run.trial116.stdout:(38/42): python3-portend-3.1.0-2.el9.noarch.rpm 294 kB/s | 16 kB 00:00 2026-02-12T18:12:52.097 INFO:teuthology.orchestra.run.trial116.stdout:(39/42): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.4 MB/s | 90 kB 00:00 2026-02-12T18:12:52.112 INFO:teuthology.orchestra.run.trial116.stdout:(40/42): python3-tempora-5.0.0-2.el9.noarch.rpm 647 kB/s | 36 kB 00:00 2026-02-12T18:12:52.149 INFO:teuthology.orchestra.run.trial116.stdout:(41/42): python3-more-itertools-8.12.0-2.el9.no 528 kB/s | 79 kB 00:00 2026-02-12T18:12:52.151 INFO:teuthology.orchestra.run.trial116.stdout:(42/42): python3-zc-lockfile-2.0-10.el9.noarch. 369 kB/s | 20 kB 00:00 2026-02-12T18:12:52.152 INFO:teuthology.orchestra.run.trial116.stdout:-------------------------------------------------------------------------------- 2026-02-12T18:12:52.152 INFO:teuthology.orchestra.run.trial116.stdout:Total 14 MB/s | 48 MB 00:03 2026-02-12T18:12:52.293 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction check 2026-02-12T18:12:52.311 INFO:teuthology.orchestra.run.trial116.stdout:Transaction check succeeded. 2026-02-12T18:12:52.311 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction test 2026-02-12T18:12:52.507 INFO:teuthology.orchestra.run.trial116.stdout:Transaction test succeeded. 2026-02-12T18:12:52.507 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction 2026-02-12T18:12:52.737 INFO:teuthology.orchestra.run.trial116.stdout: Preparing : 1/1 2026-02-12T18:12:52.738 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/42 2026-02-12T18:12:52.743 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/42 2026-02-12T18:12:52.748 INFO:teuthology.orchestra.run.trial116.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/42 2026-02-12T18:12:52.750 INFO:teuthology.orchestra.run.trial116.stdout: Installing : unzip-6.0-59.el9.x86_64 4/42 2026-02-12T18:12:52.755 INFO:teuthology.orchestra.run.trial116.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 5/42 2026-02-12T18:12:52.779 INFO:teuthology.orchestra.run.trial116.stdout: Installing : zip-3.0-35.el9.x86_64 6/42 2026-02-12T18:12:52.782 INFO:teuthology.orchestra.run.trial116.stdout: Installing : luarocks-3.9.2-5.el9.noarch 7/42 2026-02-12T18:12:52.787 INFO:teuthology.orchestra.run.trial116.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 8/42 2026-02-12T18:12:52.792 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 9/42 2026-02-12T18:12:52.798 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 10/42 2026-02-12T18:12:52.801 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 11/42 2026-02-12T18:12:52.807 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 12/42 2026-02-12T18:12:52.812 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 13/42 2026-02-12T18:12:52.816 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 14/42 2026-02-12T18:12:52.822 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 15/42 2026-02-12T18:12:52.825 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 16/42 2026-02-12T18:12:52.832 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 17/42 2026-02-12T18:12:52.835 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 18/42 2026-02-12T18:12:52.858 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 19/42 2026-02-12T18:12:52.870 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 20/42 2026-02-12T18:12:52.875 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 21/42 2026-02-12T18:12:52.885 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 22/42 2026-02-12T18:12:52.892 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-packaging-20.9-5.el9.noarch 23/42 2026-02-12T18:12:52.906 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/42 2026-02-12T18:12:52.921 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/42 2026-02-12T18:12:52.996 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/42 2026-02-12T18:12:53.004 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 27/42 2026-02-12T18:12:53.027 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/42 2026-02-12T18:12:53.056 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 29/42 2026-02-12T18:12:53.062 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/42 2026-02-12T18:12:53.113 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/42 2026-02-12T18:12:53.154 INFO:teuthology.orchestra.run.trial116.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5230.gafefaa40.el9. 32/42 2026-02-12T18:12:53.155 INFO:teuthology.orchestra.run.trial116.stdout: Installing : ceph-mgr-2:20.3.0-5230.gafefaa40.el9.x86_64 33/42 2026-02-12T18:12:53.178 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5230.gafefaa40.el9.x86_64 33/42 2026-02-12T18:12:53.178 INFO:teuthology.orchestra.run.trial116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T18:12:53.178 INFO:teuthology.orchestra.run.trial116.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-12T18:12:53.178 INFO:teuthology.orchestra.run.trial116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-12T18:12:53.178 INFO:teuthology.orchestra.run.trial116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-12T18:12:53.179 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:53.183 INFO:teuthology.orchestra.run.trial116.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 34/42 2026-02-12T18:12:53.200 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-12T18:12:53.200 INFO:teuthology.orchestra.run.trial116.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-12T18:12:53.200 INFO:teuthology.orchestra.run.trial116.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-12T18:12:53.200 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:53.210 INFO:teuthology.orchestra.run.trial116.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-12T18:12:53.233 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-12T18:12:53.233 INFO:teuthology.orchestra.run.trial116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-12T18:12:53.233 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:53.864 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 36/42 2026-02-12T18:12:54.624 INFO:teuthology.orchestra.run.trial116.stdout: Installing : ceph-osd-classic-2:20.3.0-5230.gafefaa40.el9.x86_6 37/42 2026-02-12T18:12:54.634 INFO:teuthology.orchestra.run.trial116.stdout: Installing : ceph-osd-crimson-2:20.3.0-5230.gafefaa40.el9.x86_6 38/42 2026-02-12T18:12:54.635 INFO:teuthology.orchestra.run.trial116.stdout: Installing : ceph-osd-2:20.3.0-5230.gafefaa40.el9.x86_64 39/42 2026-02-12T18:12:54.659 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: ceph-osd-2:20.3.0-5230.gafefaa40.el9.x86_64 39/42 2026-02-12T18:12:54.659 INFO:teuthology.orchestra.run.trial116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T18:12:54.659 INFO:teuthology.orchestra.run.trial116.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-12T18:12:54.659 INFO:teuthology.orchestra.run.trial116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-12T18:12:54.659 INFO:teuthology.orchestra.run.trial116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-12T18:12:54.659 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:54.870 INFO:teuthology.orchestra.run.trial116.stdout: Installing : ceph-mon-2:20.3.0-5230.gafefaa40.el9.x86_64 40/42 2026-02-12T18:12:54.891 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: ceph-mon-2:20.3.0-5230.gafefaa40.el9.x86_64 40/42 2026-02-12T18:12:54.891 INFO:teuthology.orchestra.run.trial116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T18:12:54.892 INFO:teuthology.orchestra.run.trial116.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-12T18:12:54.892 INFO:teuthology.orchestra.run.trial116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-12T18:12:54.892 INFO:teuthology.orchestra.run.trial116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-12T18:12:54.892 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:54.986 INFO:teuthology.orchestra.run.trial116.stdout: Installing : ceph-mds-2:20.3.0-5230.gafefaa40.el9.x86_64 41/42 2026-02-12T18:12:55.006 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: ceph-mds-2:20.3.0-5230.gafefaa40.el9.x86_64 41/42 2026-02-12T18:12:55.007 INFO:teuthology.orchestra.run.trial116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T18:12:55.007 INFO:teuthology.orchestra.run.trial116.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-12T18:12:55.007 INFO:teuthology.orchestra.run.trial116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-12T18:12:55.007 INFO:teuthology.orchestra.run.trial116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-12T18:12:55.007 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:55.008 INFO:teuthology.orchestra.run.trial116.stdout: Installing : ceph-2:20.3.0-5230.gafefaa40.el9.x86_64 42/42 2026-02-12T18:12:55.014 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5230.gafefaa40.el9.x86_6 42/42 2026-02-12T18:12:55.037 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5230.gafefaa40.el9.x86_6 42/42 2026-02-12T18:12:55.411 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: ceph-2:20.3.0-5230.gafefaa40.el9.x86_64 42/42 2026-02-12T18:12:55.411 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : ceph-2:20.3.0-5230.gafefaa40.el9.x86_64 1/42 2026-02-12T18:12:55.412 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : ceph-mds-2:20.3.0-5230.gafefaa40.el9.x86_64 2/42 2026-02-12T18:12:55.412 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : ceph-mgr-2:20.3.0-5230.gafefaa40.el9.x86_64 3/42 2026-02-12T18:12:55.412 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : ceph-mon-2:20.3.0-5230.gafefaa40.el9.x86_64 4/42 2026-02-12T18:12:55.412 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : ceph-osd-2:20.3.0-5230.gafefaa40.el9.x86_64 5/42 2026-02-12T18:12:55.412 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : ceph-osd-classic-2:20.3.0-5230.gafefaa40.el9.x86_6 6/42 2026-02-12T18:12:55.412 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5230.gafefaa40.el9.x86_6 7/42 2026-02-12T18:12:55.412 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5230.gafefaa40.el9. 8/42 2026-02-12T18:12:55.412 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 9/42 2026-02-12T18:12:55.412 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 10/42 2026-02-12T18:12:55.412 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/42 2026-02-12T18:12:55.412 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 12/42 2026-02-12T18:12:55.412 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/42 2026-02-12T18:12:55.413 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/42 2026-02-12T18:12:55.413 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 15/42 2026-02-12T18:12:55.413 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 16/42 2026-02-12T18:12:55.413 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 17/42 2026-02-12T18:12:55.413 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : unzip-6.0-59.el9.x86_64 18/42 2026-02-12T18:12:55.413 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : zip-3.0-35.el9.x86_64 19/42 2026-02-12T18:12:55.413 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 20/42 2026-02-12T18:12:55.413 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : lua-5.4.4-4.el9.x86_64 21/42 2026-02-12T18:12:55.413 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 22/42 2026-02-12T18:12:55.413 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 23/42 2026-02-12T18:12:55.413 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 24/42 2026-02-12T18:12:55.413 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 25/42 2026-02-12T18:12:55.414 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 26/42 2026-02-12T18:12:55.414 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 27/42 2026-02-12T18:12:55.414 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 28/42 2026-02-12T18:12:55.414 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 29/42 2026-02-12T18:12:55.414 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 30/42 2026-02-12T18:12:55.414 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 31/42 2026-02-12T18:12:55.414 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 32/42 2026-02-12T18:12:55.414 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 33/42 2026-02-12T18:12:55.414 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 34/42 2026-02-12T18:12:55.414 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 35/42 2026-02-12T18:12:55.414 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 36/42 2026-02-12T18:12:55.414 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 37/42 2026-02-12T18:12:55.414 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 38/42 2026-02-12T18:12:55.415 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 39/42 2026-02-12T18:12:55.415 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 40/42 2026-02-12T18:12:55.415 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 41/42 2026-02-12T18:12:55.469 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 42/42 2026-02-12T18:12:55.469 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:55.469 INFO:teuthology.orchestra.run.trial116.stdout:Installed: 2026-02-12T18:12:55.469 INFO:teuthology.orchestra.run.trial116.stdout: ceph-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:12:55.469 INFO:teuthology.orchestra.run.trial116.stdout: ceph-mds-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:12:55.469 INFO:teuthology.orchestra.run.trial116.stdout: ceph-mgr-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:12:55.469 INFO:teuthology.orchestra.run.trial116.stdout: ceph-mgr-modules-core-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-12T18:12:55.469 INFO:teuthology.orchestra.run.trial116.stdout: ceph-mon-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:12:55.470 INFO:teuthology.orchestra.run.trial116.stdout: ceph-osd-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:12:55.470 INFO:teuthology.orchestra.run.trial116.stdout: ceph-osd-classic-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:12:55.470 INFO:teuthology.orchestra.run.trial116.stdout: ceph-osd-crimson-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:12:55.470 INFO:teuthology.orchestra.run.trial116.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-12T18:12:55.470 INFO:teuthology.orchestra.run.trial116.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-12T18:12:55.470 INFO:teuthology.orchestra.run.trial116.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-12T18:12:55.470 INFO:teuthology.orchestra.run.trial116.stdout: lua-5.4.4-4.el9.x86_64 2026-02-12T18:12:55.470 INFO:teuthology.orchestra.run.trial116.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-12T18:12:55.470 INFO:teuthology.orchestra.run.trial116.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-12T18:12:55.470 INFO:teuthology.orchestra.run.trial116.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-12T18:12:55.470 INFO:teuthology.orchestra.run.trial116.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-12T18:12:55.470 INFO:teuthology.orchestra.run.trial116.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-12T18:12:55.471 INFO:teuthology.orchestra.run.trial116.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-12T18:12:55.471 INFO:teuthology.orchestra.run.trial116.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-12T18:12:55.471 INFO:teuthology.orchestra.run.trial116.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-12T18:12:55.471 INFO:teuthology.orchestra.run.trial116.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-12T18:12:55.471 INFO:teuthology.orchestra.run.trial116.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-12T18:12:55.471 INFO:teuthology.orchestra.run.trial116.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-12T18:12:55.471 INFO:teuthology.orchestra.run.trial116.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-12T18:12:55.471 INFO:teuthology.orchestra.run.trial116.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-12T18:12:55.471 INFO:teuthology.orchestra.run.trial116.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-12T18:12:55.471 INFO:teuthology.orchestra.run.trial116.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-12T18:12:55.471 INFO:teuthology.orchestra.run.trial116.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-12T18:12:55.471 INFO:teuthology.orchestra.run.trial116.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-12T18:12:55.472 INFO:teuthology.orchestra.run.trial116.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-12T18:12:55.472 INFO:teuthology.orchestra.run.trial116.stdout: python3-ply-3.11-14.el9.noarch 2026-02-12T18:12:55.472 INFO:teuthology.orchestra.run.trial116.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-12T18:12:55.472 INFO:teuthology.orchestra.run.trial116.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-12T18:12:55.472 INFO:teuthology.orchestra.run.trial116.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-12T18:12:55.472 INFO:teuthology.orchestra.run.trial116.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-12T18:12:55.472 INFO:teuthology.orchestra.run.trial116.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-12T18:12:55.472 INFO:teuthology.orchestra.run.trial116.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-12T18:12:55.472 INFO:teuthology.orchestra.run.trial116.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-12T18:12:55.472 INFO:teuthology.orchestra.run.trial116.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-12T18:12:55.472 INFO:teuthology.orchestra.run.trial116.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-12T18:12:55.472 INFO:teuthology.orchestra.run.trial116.stdout: unzip-6.0-59.el9.x86_64 2026-02-12T18:12:55.472 INFO:teuthology.orchestra.run.trial116.stdout: zip-3.0-35.el9.x86_64 2026-02-12T18:12:55.473 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:55.473 INFO:teuthology.orchestra.run.trial116.stdout:Complete! 2026-02-12T18:12:55.507 DEBUG:teuthology.orchestra.run.trial116:> sudo yum -y install ceph-base 2026-02-12T18:12:55.810 INFO:teuthology.orchestra.run.trial116.stdout:Last metadata expiration check: 0:00:29 ago on Thu 12 Feb 2026 06:12:26 PM UTC. 2026-02-12T18:12:55.876 INFO:teuthology.orchestra.run.trial116.stdout:Package ceph-base-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-12T18:12:55.893 INFO:teuthology.orchestra.run.trial116.stdout:Dependencies resolved. 2026-02-12T18:12:55.894 INFO:teuthology.orchestra.run.trial116.stdout:Nothing to do. 2026-02-12T18:12:55.894 INFO:teuthology.orchestra.run.trial116.stdout:Complete! 2026-02-12T18:12:55.915 DEBUG:teuthology.orchestra.run.trial116:> sudo yum -y install cephadm 2026-02-12T18:12:56.214 INFO:teuthology.orchestra.run.trial116.stdout:Last metadata expiration check: 0:00:30 ago on Thu 12 Feb 2026 06:12:26 PM UTC. 2026-02-12T18:12:56.297 INFO:teuthology.orchestra.run.trial116.stdout:Dependencies resolved. 2026-02-12T18:12:56.298 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:12:56.298 INFO:teuthology.orchestra.run.trial116.stdout: Package Arch Version Repository Size 2026-02-12T18:12:56.298 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:12:56.298 INFO:teuthology.orchestra.run.trial116.stdout:Installing: 2026-02-12T18:12:56.298 INFO:teuthology.orchestra.run.trial116.stdout: cephadm noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 1.0 M 2026-02-12T18:12:56.298 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:56.298 INFO:teuthology.orchestra.run.trial116.stdout:Transaction Summary 2026-02-12T18:12:56.299 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:12:56.299 INFO:teuthology.orchestra.run.trial116.stdout:Install 1 Package 2026-02-12T18:12:56.299 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:56.299 INFO:teuthology.orchestra.run.trial116.stdout:Total download size: 1.0 M 2026-02-12T18:12:56.299 INFO:teuthology.orchestra.run.trial116.stdout:Installed size: 1.0 M 2026-02-12T18:12:56.299 INFO:teuthology.orchestra.run.trial116.stdout:Downloading Packages: 2026-02-12T18:12:56.419 INFO:teuthology.orchestra.run.trial116.stdout:cephadm-20.3.0-5230.gafefaa40.el9.noarch.rpm 8.5 MB/s | 1.0 MB 00:00 2026-02-12T18:12:56.419 INFO:teuthology.orchestra.run.trial116.stdout:-------------------------------------------------------------------------------- 2026-02-12T18:12:56.419 INFO:teuthology.orchestra.run.trial116.stdout:Total 8.4 MB/s | 1.0 MB 00:00 2026-02-12T18:12:56.419 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction check 2026-02-12T18:12:56.421 INFO:teuthology.orchestra.run.trial116.stdout:Transaction check succeeded. 2026-02-12T18:12:56.421 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction test 2026-02-12T18:12:56.425 INFO:teuthology.orchestra.run.trial116.stdout:Transaction test succeeded. 2026-02-12T18:12:56.425 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction 2026-02-12T18:12:56.443 INFO:teuthology.orchestra.run.trial116.stdout: Preparing : 1/1 2026-02-12T18:12:56.506 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 1/1 2026-02-12T18:12:56.520 INFO:teuthology.orchestra.run.trial116.stdout: Installing : cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 1/1 2026-02-12T18:12:56.584 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 1/1 2026-02-12T18:12:56.639 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 1/1 2026-02-12T18:12:56.639 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:56.639 INFO:teuthology.orchestra.run.trial116.stdout:Installed: 2026-02-12T18:12:56.639 INFO:teuthology.orchestra.run.trial116.stdout: cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-12T18:12:56.639 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:56.639 INFO:teuthology.orchestra.run.trial116.stdout:Complete! 2026-02-12T18:12:56.667 DEBUG:teuthology.orchestra.run.trial116:> sudo yum -y install ceph-immutable-object-cache 2026-02-12T18:12:56.964 INFO:teuthology.orchestra.run.trial116.stdout:Last metadata expiration check: 0:00:30 ago on Thu 12 Feb 2026 06:12:26 PM UTC. 2026-02-12T18:12:57.047 INFO:teuthology.orchestra.run.trial116.stdout:Dependencies resolved. 2026-02-12T18:12:57.047 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:12:57.047 INFO:teuthology.orchestra.run.trial116.stdout: Package Arch Version Repo Size 2026-02-12T18:12:57.048 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:12:57.048 INFO:teuthology.orchestra.run.trial116.stdout:Installing: 2026-02-12T18:12:57.048 INFO:teuthology.orchestra.run.trial116.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 147 k 2026-02-12T18:12:57.048 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:57.048 INFO:teuthology.orchestra.run.trial116.stdout:Transaction Summary 2026-02-12T18:12:57.048 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:12:57.048 INFO:teuthology.orchestra.run.trial116.stdout:Install 1 Package 2026-02-12T18:12:57.048 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:57.048 INFO:teuthology.orchestra.run.trial116.stdout:Total download size: 147 k 2026-02-12T18:12:57.048 INFO:teuthology.orchestra.run.trial116.stdout:Installed size: 419 k 2026-02-12T18:12:57.048 INFO:teuthology.orchestra.run.trial116.stdout:Downloading Packages: 2026-02-12T18:12:57.133 INFO:teuthology.orchestra.run.trial116.stdout:ceph-immutable-object-cache-20.3.0-5230.gafefaa 1.7 MB/s | 147 kB 00:00 2026-02-12T18:12:57.133 INFO:teuthology.orchestra.run.trial116.stdout:-------------------------------------------------------------------------------- 2026-02-12T18:12:57.133 INFO:teuthology.orchestra.run.trial116.stdout:Total 1.7 MB/s | 147 kB 00:00 2026-02-12T18:12:57.133 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction check 2026-02-12T18:12:57.137 INFO:teuthology.orchestra.run.trial116.stdout:Transaction check succeeded. 2026-02-12T18:12:57.137 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction test 2026-02-12T18:12:57.164 INFO:teuthology.orchestra.run.trial116.stdout:Transaction test succeeded. 2026-02-12T18:12:57.164 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction 2026-02-12T18:12:57.213 INFO:teuthology.orchestra.run.trial116.stdout: Preparing : 1/1 2026-02-12T18:12:57.214 INFO:teuthology.orchestra.run.trial116.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5230.gafefaa40. 1/1 2026-02-12T18:12:57.236 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5230.gafefaa40. 1/1 2026-02-12T18:12:57.236 INFO:teuthology.orchestra.run.trial116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T18:12:57.236 INFO:teuthology.orchestra.run.trial116.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-12T18:12:57.236 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:57.508 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5230.gafefaa40. 1/1 2026-02-12T18:12:57.508 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:57.508 INFO:teuthology.orchestra.run.trial116.stdout:Installed: 2026-02-12T18:12:57.509 INFO:teuthology.orchestra.run.trial116.stdout: ceph-immutable-object-cache-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:12:57.509 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:57.509 INFO:teuthology.orchestra.run.trial116.stdout:Complete! 2026-02-12T18:12:57.535 DEBUG:teuthology.orchestra.run.trial116:> sudo yum -y install ceph-mgr 2026-02-12T18:12:57.835 INFO:teuthology.orchestra.run.trial116.stdout:Last metadata expiration check: 0:00:31 ago on Thu 12 Feb 2026 06:12:26 PM UTC. 2026-02-12T18:12:57.900 INFO:teuthology.orchestra.run.trial116.stdout:Package ceph-mgr-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-12T18:12:57.918 INFO:teuthology.orchestra.run.trial116.stdout:Dependencies resolved. 2026-02-12T18:12:57.918 INFO:teuthology.orchestra.run.trial116.stdout:Nothing to do. 2026-02-12T18:12:57.918 INFO:teuthology.orchestra.run.trial116.stdout:Complete! 2026-02-12T18:12:57.939 DEBUG:teuthology.orchestra.run.trial116:> sudo yum -y install ceph-mgr-dashboard 2026-02-12T18:12:58.235 INFO:teuthology.orchestra.run.trial116.stdout:Last metadata expiration check: 0:00:32 ago on Thu 12 Feb 2026 06:12:26 PM UTC. 2026-02-12T18:12:58.318 INFO:teuthology.orchestra.run.trial116.stdout:Dependencies resolved. 2026-02-12T18:12:58.319 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:12:58.319 INFO:teuthology.orchestra.run.trial116.stdout: Package Arch Version Repository Size 2026-02-12T18:12:58.319 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:12:58.319 INFO:teuthology.orchestra.run.trial116.stdout:Installing: 2026-02-12T18:12:58.319 INFO:teuthology.orchestra.run.trial116.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 51 M 2026-02-12T18:12:58.319 INFO:teuthology.orchestra.run.trial116.stdout:Installing dependencies: 2026-02-12T18:12:58.319 INFO:teuthology.orchestra.run.trial116.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-12T18:12:58.319 INFO:teuthology.orchestra.run.trial116.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 57 k 2026-02-12T18:12:58.319 INFO:teuthology.orchestra.run.trial116.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 17 k 2026-02-12T18:12:58.320 INFO:teuthology.orchestra.run.trial116.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-12T18:12:58.320 INFO:teuthology.orchestra.run.trial116.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-12T18:12:58.320 INFO:teuthology.orchestra.run.trial116.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-12T18:12:58.320 INFO:teuthology.orchestra.run.trial116.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-12T18:12:58.320 INFO:teuthology.orchestra.run.trial116.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-12T18:12:58.320 INFO:teuthology.orchestra.run.trial116.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-12T18:12:58.320 INFO:teuthology.orchestra.run.trial116.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-12T18:12:58.320 INFO:teuthology.orchestra.run.trial116.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-12T18:12:58.320 INFO:teuthology.orchestra.run.trial116.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-12T18:12:58.320 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:58.320 INFO:teuthology.orchestra.run.trial116.stdout:Transaction Summary 2026-02-12T18:12:58.321 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:12:58.321 INFO:teuthology.orchestra.run.trial116.stdout:Install 13 Packages 2026-02-12T18:12:58.321 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:12:58.321 INFO:teuthology.orchestra.run.trial116.stdout:Total download size: 55 M 2026-02-12T18:12:58.321 INFO:teuthology.orchestra.run.trial116.stdout:Installed size: 262 M 2026-02-12T18:12:58.322 INFO:teuthology.orchestra.run.trial116.stdout:Downloading Packages: 2026-02-12T18:12:58.675 INFO:teuthology.orchestra.run.trial116.stdout:(1/13): ceph-prometheus-alerts-20.3.0-5230.gafe 250 kB/s | 17 kB 00:00 2026-02-12T18:12:58.677 INFO:teuthology.orchestra.run.trial116.stdout:(2/13): ceph-grafana-dashboards-20.3.0-5230.gaf 799 kB/s | 57 kB 00:00 2026-02-12T18:12:58.746 INFO:teuthology.orchestra.run.trial116.stdout:(3/13): python3-jmespath-1.0.1-1.el9.noarch.rpm 671 kB/s | 48 kB 00:00 2026-02-12T18:12:58.786 INFO:teuthology.orchestra.run.trial116.stdout:(4/13): python3-protobuf-3.14.0-17.el9.noarch.r 2.4 MB/s | 267 kB 00:00 2026-02-12T18:12:58.877 INFO:teuthology.orchestra.run.trial116.stdout:(5/13): protobuf-compiler-3.14.0-17.el9.x86_64. 6.4 MB/s | 862 kB 00:00 2026-02-12T18:12:59.278 INFO:teuthology.orchestra.run.trial116.stdout:(6/13): grpc-data-1.46.7-10.el9.noarch.rpm 48 kB/s | 19 kB 00:00 2026-02-12T18:12:59.643 INFO:teuthology.orchestra.run.trial116.stdout:(7/13): ceph-mgr-dashboard-20.3.0-5230.gafefaa4 49 MB/s | 51 MB 00:01 2026-02-12T18:12:59.808 INFO:teuthology.orchestra.run.trial116.stdout:(8/13): abseil-cpp-20211102.0-4.el9.x86_64.rpm 540 kB/s | 551 kB 00:01 2026-02-12T18:12:59.888 INFO:teuthology.orchestra.run.trial116.stdout:(9/13): python3-repoze-lru-0.7-16.el9.noarch.rp 386 kB/s | 31 kB 00:00 2026-02-12T18:13:00.694 INFO:teuthology.orchestra.run.trial116.stdout:(10/13): python3-grpcio-tools-1.46.7-10.el9.x86 138 kB/s | 144 kB 00:01 2026-02-12T18:13:00.712 INFO:teuthology.orchestra.run.trial116.stdout:(11/13): python3-routes-2.5.1-5.el9.noarch.rpm 229 kB/s | 188 kB 00:00 2026-02-12T18:13:00.773 INFO:teuthology.orchestra.run.trial116.stdout:(12/13): python3-xmltodict-0.12.0-15.el9.noarch 277 kB/s | 22 kB 00:00 2026-02-12T18:13:01.976 INFO:teuthology.orchestra.run.trial116.stdout:(13/13): python3-grpcio-1.46.7-10.el9.x86_64.rp 773 kB/s | 2.0 MB 00:02 2026-02-12T18:13:01.981 INFO:teuthology.orchestra.run.trial116.stdout:-------------------------------------------------------------------------------- 2026-02-12T18:13:01.982 INFO:teuthology.orchestra.run.trial116.stdout:Total 15 MB/s | 55 MB 00:03 2026-02-12T18:13:02.009 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction check 2026-02-12T18:13:02.019 INFO:teuthology.orchestra.run.trial116.stdout:Transaction check succeeded. 2026-02-12T18:13:02.019 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction test 2026-02-12T18:13:02.232 INFO:teuthology.orchestra.run.trial116.stdout:Transaction test succeeded. 2026-02-12T18:13:02.232 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction 2026-02-12T18:13:02.458 INFO:teuthology.orchestra.run.trial116.stdout: Preparing : 1/1 2026-02-12T18:13:02.464 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 1/13 2026-02-12T18:13:02.475 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 2/13 2026-02-12T18:13:02.477 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 3/13 2026-02-12T18:13:02.504 INFO:teuthology.orchestra.run.trial116.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 4/13 2026-02-12T18:13:02.547 INFO:teuthology.orchestra.run.trial116.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 5/13 2026-02-12T18:13:02.556 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 6/13 2026-02-12T18:13:02.583 INFO:teuthology.orchestra.run.trial116.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 7/13 2026-02-12T18:13:02.593 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 8/13 2026-02-12T18:13:02.600 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/13 2026-02-12T18:13:02.603 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 10/13 2026-02-12T18:13:02.612 INFO:teuthology.orchestra.run.trial116.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5230.gafefaa40.el9 11/13 2026-02-12T18:13:05.485 INFO:teuthology.orchestra.run.trial116.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5230.gafefaa40.el 12/13 2026-02-12T18:13:05.492 INFO:teuthology.orchestra.run.trial116.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5230.gafefaa40.el9.noa 13/13 2026-02-12T18:13:05.746 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5230.gafefaa40.el9.noa 13/13 2026-02-12T18:13:05.746 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5230.gafefaa40.el 1/13 2026-02-12T18:13:05.746 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5230.gafefaa40.el9.noa 2/13 2026-02-12T18:13:05.746 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5230.gafefaa40.el9 3/13 2026-02-12T18:13:05.747 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 4/13 2026-02-12T18:13:05.747 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 5/13 2026-02-12T18:13:05.747 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 6/13 2026-02-12T18:13:05.747 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 7/13 2026-02-12T18:13:05.747 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/13 2026-02-12T18:13:05.747 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/13 2026-02-12T18:13:05.747 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/13 2026-02-12T18:13:05.747 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/13 2026-02-12T18:13:05.747 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/13 2026-02-12T18:13:05.802 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 13/13 2026-02-12T18:13:05.803 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:05.803 INFO:teuthology.orchestra.run.trial116.stdout:Installed: 2026-02-12T18:13:05.803 INFO:teuthology.orchestra.run.trial116.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-12T18:13:05.803 INFO:teuthology.orchestra.run.trial116.stdout: ceph-grafana-dashboards-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-12T18:13:05.803 INFO:teuthology.orchestra.run.trial116.stdout: ceph-mgr-dashboard-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-12T18:13:05.803 INFO:teuthology.orchestra.run.trial116.stdout: ceph-prometheus-alerts-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-12T18:13:05.803 INFO:teuthology.orchestra.run.trial116.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-12T18:13:05.803 INFO:teuthology.orchestra.run.trial116.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-12T18:13:05.803 INFO:teuthology.orchestra.run.trial116.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-12T18:13:05.803 INFO:teuthology.orchestra.run.trial116.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-12T18:13:05.803 INFO:teuthology.orchestra.run.trial116.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-12T18:13:05.804 INFO:teuthology.orchestra.run.trial116.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-12T18:13:05.804 INFO:teuthology.orchestra.run.trial116.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-12T18:13:05.804 INFO:teuthology.orchestra.run.trial116.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-12T18:13:05.804 INFO:teuthology.orchestra.run.trial116.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-12T18:13:05.804 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:05.804 INFO:teuthology.orchestra.run.trial116.stdout:Complete! 2026-02-12T18:13:05.840 DEBUG:teuthology.orchestra.run.trial116:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-12T18:13:06.137 INFO:teuthology.orchestra.run.trial116.stdout:Last metadata expiration check: 0:00:40 ago on Thu 12 Feb 2026 06:12:26 PM UTC. 2026-02-12T18:13:06.221 INFO:teuthology.orchestra.run.trial116.stdout:Dependencies resolved. 2026-02-12T18:13:06.222 INFO:teuthology.orchestra.run.trial116.stdout:======================================================================================= 2026-02-12T18:13:06.222 INFO:teuthology.orchestra.run.trial116.stdout: Package Arch Version Repository Size 2026-02-12T18:13:06.222 INFO:teuthology.orchestra.run.trial116.stdout:======================================================================================= 2026-02-12T18:13:06.222 INFO:teuthology.orchestra.run.trial116.stdout:Installing: 2026-02-12T18:13:06.222 INFO:teuthology.orchestra.run.trial116.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 7.4 M 2026-02-12T18:13:06.222 INFO:teuthology.orchestra.run.trial116.stdout:Installing dependencies: 2026-02-12T18:13:06.222 INFO:teuthology.orchestra.run.trial116.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2026-02-12T18:13:06.222 INFO:teuthology.orchestra.run.trial116.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2026-02-12T18:13:06.222 INFO:teuthology.orchestra.run.trial116.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2026-02-12T18:13:06.223 INFO:teuthology.orchestra.run.trial116.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-12T18:13:06.223 INFO:teuthology.orchestra.run.trial116.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-12T18:13:06.223 INFO:teuthology.orchestra.run.trial116.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-12T18:13:06.223 INFO:teuthology.orchestra.run.trial116.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-12T18:13:06.223 INFO:teuthology.orchestra.run.trial116.stdout: python3-devel x86_64 3.9.25-2.el9 appstream 244 k 2026-02-12T18:13:06.223 INFO:teuthology.orchestra.run.trial116.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-12T18:13:06.223 INFO:teuthology.orchestra.run.trial116.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-12T18:13:06.223 INFO:teuthology.orchestra.run.trial116.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-12T18:13:06.223 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:06.223 INFO:teuthology.orchestra.run.trial116.stdout:Transaction Summary 2026-02-12T18:13:06.223 INFO:teuthology.orchestra.run.trial116.stdout:======================================================================================= 2026-02-12T18:13:06.223 INFO:teuthology.orchestra.run.trial116.stdout:Install 12 Packages 2026-02-12T18:13:06.224 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:06.224 INFO:teuthology.orchestra.run.trial116.stdout:Total download size: 43 M 2026-02-12T18:13:06.224 INFO:teuthology.orchestra.run.trial116.stdout:Installed size: 234 M 2026-02-12T18:13:06.224 INFO:teuthology.orchestra.run.trial116.stdout:Downloading Packages: 2026-02-12T18:13:07.106 INFO:teuthology.orchestra.run.trial116.stdout:(1/12): ceph-mgr-diskprediction-local-20.3.0-52 33 MB/s | 7.4 MB 00:00 2026-02-12T18:13:07.170 INFO:teuthology.orchestra.run.trial116.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 514 kB/s | 33 kB 00:00 2026-02-12T18:13:07.232 INFO:teuthology.orchestra.run.trial116.stdout:(3/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 48 MB/s | 3.0 MB 00:00 2026-02-12T18:13:07.270 INFO:teuthology.orchestra.run.trial116.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 478 kB/s | 18 kB 00:00 2026-02-12T18:13:07.316 INFO:teuthology.orchestra.run.trial116.stdout:(5/12): openblas-0.3.29-1.el9.x86_64.rpm 910 kB/s | 42 kB 00:00 2026-02-12T18:13:07.348 INFO:teuthology.orchestra.run.trial116.stdout:(6/12): libquadmath-11.5.0-14.el9.x86_64.rpm 392 kB/s | 184 kB 00:00 2026-02-12T18:13:07.389 INFO:teuthology.orchestra.run.trial116.stdout:(7/12): openblas-openmp-0.3.29-1.el9.x86_64.rpm 73 MB/s | 5.3 MB 00:00 2026-02-12T18:13:07.396 INFO:teuthology.orchestra.run.trial116.stdout:(8/12): python3-devel-3.9.25-2.el9.x86_64.rpm 5.0 MB/s | 244 kB 00:00 2026-02-12T18:13:07.432 INFO:teuthology.orchestra.run.trial116.stdout:(9/12): python3-numpy-f2py-1.23.5-2.el9.x86_64. 12 MB/s | 442 kB 00:00 2026-02-12T18:13:07.463 INFO:teuthology.orchestra.run.trial116.stdout:(10/12): python3-numpy-1.23.5-2.el9.x86_64.rpm 82 MB/s | 6.1 MB 00:00 2026-02-12T18:13:07.509 INFO:teuthology.orchestra.run.trial116.stdout:(11/12): libgfortran-11.5.0-14.el9.x86_64.rpm 1.2 MB/s | 794 kB 00:00 2026-02-12T18:13:07.614 INFO:teuthology.orchestra.run.trial116.stdout:(12/12): python3-scipy-1.9.3-2.el9.x86_64.rpm 106 MB/s | 19 MB 00:00 2026-02-12T18:13:07.616 INFO:teuthology.orchestra.run.trial116.stdout:-------------------------------------------------------------------------------- 2026-02-12T18:13:07.616 INFO:teuthology.orchestra.run.trial116.stdout:Total 31 MB/s | 43 MB 00:01 2026-02-12T18:13:07.771 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction check 2026-02-12T18:13:07.785 INFO:teuthology.orchestra.run.trial116.stdout:Transaction check succeeded. 2026-02-12T18:13:07.785 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction test 2026-02-12T18:13:07.989 INFO:teuthology.orchestra.run.trial116.stdout:Transaction test succeeded. 2026-02-12T18:13:07.989 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction 2026-02-12T18:13:08.209 INFO:teuthology.orchestra.run.trial116.stdout: Preparing : 1/1 2026-02-12T18:13:08.212 INFO:teuthology.orchestra.run.trial116.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2026-02-12T18:13:08.218 INFO:teuthology.orchestra.run.trial116.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 2/12 2026-02-12T18:13:08.251 INFO:teuthology.orchestra.run.trial116.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 3/12 2026-02-12T18:13:08.256 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-devel-3.9.25-2.el9.x86_64 4/12 2026-02-12T18:13:08.315 INFO:teuthology.orchestra.run.trial116.stdout: Installing : openblas-0.3.29-1.el9.x86_64 5/12 2026-02-12T18:13:08.317 INFO:teuthology.orchestra.run.trial116.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 6/12 2026-02-12T18:13:08.338 INFO:teuthology.orchestra.run.trial116.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2026-02-12T18:13:08.659 INFO:teuthology.orchestra.run.trial116.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2026-02-12T18:13:08.731 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 9/12 2026-02-12T18:13:09.391 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 10/12 2026-02-12T18:13:09.849 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 11/12 2026-02-12T18:13:09.849 INFO:teuthology.orchestra.run.trial116.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5230.gafefa 12/12 2026-02-12T18:13:10.123 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5230.gafefa 12/12 2026-02-12T18:13:10.123 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5230.gafefa 1/12 2026-02-12T18:13:10.124 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 2/12 2026-02-12T18:13:10.124 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 3/12 2026-02-12T18:13:10.124 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2026-02-12T18:13:10.124 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2026-02-12T18:13:10.124 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2026-02-12T18:13:10.124 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 7/12 2026-02-12T18:13:10.124 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 8/12 2026-02-12T18:13:10.124 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-devel-3.9.25-2.el9.x86_64 9/12 2026-02-12T18:13:10.124 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 10/12 2026-02-12T18:13:10.124 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 11/12 2026-02-12T18:13:10.182 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 12/12 2026-02-12T18:13:10.182 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:10.182 INFO:teuthology.orchestra.run.trial116.stdout:Installed: 2026-02-12T18:13:10.182 INFO:teuthology.orchestra.run.trial116.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-12T18:13:10.182 INFO:teuthology.orchestra.run.trial116.stdout: flexiblas-3.0.4-8.el9.x86_64 2026-02-12T18:13:10.182 INFO:teuthology.orchestra.run.trial116.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2026-02-12T18:13:10.182 INFO:teuthology.orchestra.run.trial116.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2026-02-12T18:13:10.182 INFO:teuthology.orchestra.run.trial116.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-12T18:13:10.182 INFO:teuthology.orchestra.run.trial116.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-12T18:13:10.183 INFO:teuthology.orchestra.run.trial116.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-12T18:13:10.183 INFO:teuthology.orchestra.run.trial116.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-12T18:13:10.183 INFO:teuthology.orchestra.run.trial116.stdout: python3-devel-3.9.25-2.el9.x86_64 2026-02-12T18:13:10.183 INFO:teuthology.orchestra.run.trial116.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-12T18:13:10.183 INFO:teuthology.orchestra.run.trial116.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-12T18:13:10.183 INFO:teuthology.orchestra.run.trial116.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-12T18:13:10.183 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:10.183 INFO:teuthology.orchestra.run.trial116.stdout:Complete! 2026-02-12T18:13:10.220 DEBUG:teuthology.orchestra.run.trial116:> sudo yum -y install ceph-mgr-rook 2026-02-12T18:13:10.522 INFO:teuthology.orchestra.run.trial116.stdout:Last metadata expiration check: 0:00:44 ago on Thu 12 Feb 2026 06:12:26 PM UTC. 2026-02-12T18:13:10.605 INFO:teuthology.orchestra.run.trial116.stdout:Dependencies resolved. 2026-02-12T18:13:10.605 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:13:10.605 INFO:teuthology.orchestra.run.trial116.stdout: Package Arch Version Repository Size 2026-02-12T18:13:10.605 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:13:10.605 INFO:teuthology.orchestra.run.trial116.stdout:Installing: 2026-02-12T18:13:10.606 INFO:teuthology.orchestra.run.trial116.stdout: ceph-mgr-rook noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 50 k 2026-02-12T18:13:10.606 INFO:teuthology.orchestra.run.trial116.stdout:Installing dependencies: 2026-02-12T18:13:10.606 INFO:teuthology.orchestra.run.trial116.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-12T18:13:10.606 INFO:teuthology.orchestra.run.trial116.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-12T18:13:10.606 INFO:teuthology.orchestra.run.trial116.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-12T18:13:10.606 INFO:teuthology.orchestra.run.trial116.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-12T18:13:10.606 INFO:teuthology.orchestra.run.trial116.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-12T18:13:10.606 INFO:teuthology.orchestra.run.trial116.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-12T18:13:10.606 INFO:teuthology.orchestra.run.trial116.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-12T18:13:10.606 INFO:teuthology.orchestra.run.trial116.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-12T18:13:10.606 INFO:teuthology.orchestra.run.trial116.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-12T18:13:10.606 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:10.607 INFO:teuthology.orchestra.run.trial116.stdout:Transaction Summary 2026-02-12T18:13:10.607 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:13:10.607 INFO:teuthology.orchestra.run.trial116.stdout:Install 10 Packages 2026-02-12T18:13:10.607 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:10.607 INFO:teuthology.orchestra.run.trial116.stdout:Total download size: 2.0 M 2026-02-12T18:13:10.608 INFO:teuthology.orchestra.run.trial116.stdout:Installed size: 25 M 2026-02-12T18:13:10.608 INFO:teuthology.orchestra.run.trial116.stdout:Downloading Packages: 2026-02-12T18:13:10.849 INFO:teuthology.orchestra.run.trial116.stdout:(1/10): ceph-mgr-rook-20.3.0-5230.gafefaa40.el9 687 kB/s | 50 kB 00:00 2026-02-12T18:13:11.452 INFO:teuthology.orchestra.run.trial116.stdout:(2/10): python3-requests-oauthlib-1.3.0-12.el9. 89 kB/s | 54 kB 00:00 2026-02-12T18:13:11.474 INFO:teuthology.orchestra.run.trial116.stdout:(3/10): python3-cachetools-4.2.4-1.el9.noarch.r 1.5 MB/s | 32 kB 00:00 2026-02-12T18:13:11.477 INFO:teuthology.orchestra.run.trial116.stdout:(4/10): python3-certifi-2023.05.07-4.el9.noarch 4.1 MB/s | 14 kB 00:00 2026-02-12T18:13:11.488 INFO:teuthology.orchestra.run.trial116.stdout:(5/10): python3-google-auth-2.45.0-1.el9.noarch 24 MB/s | 254 kB 00:00 2026-02-12T18:13:11.496 INFO:teuthology.orchestra.run.trial116.stdout:(6/10): python3-pyasn1-0.4.8-6.el9.noarch.rpm 220 kB/s | 159 kB 00:00 2026-02-12T18:13:11.498 INFO:teuthology.orchestra.run.trial116.stdout:(7/10): python3-kubernetes-26.1.0-3.el9.noarch. 97 MB/s | 1.0 MB 00:00 2026-02-12T18:13:11.502 INFO:teuthology.orchestra.run.trial116.stdout:(8/10): python3-websocket-client-1.2.3-2.el9.no 25 MB/s | 90 kB 00:00 2026-02-12T18:13:11.514 INFO:teuthology.orchestra.run.trial116.stdout:(9/10): python3-rsa-4.9-2.el9.noarch.rpm 3.1 MB/s | 59 kB 00:00 2026-02-12T18:13:11.555 INFO:teuthology.orchestra.run.trial116.stdout:(10/10): python3-pyasn1-modules-0.4.8-6.el9.noa 358 kB/s | 279 kB 00:00 2026-02-12T18:13:11.556 INFO:teuthology.orchestra.run.trial116.stdout:-------------------------------------------------------------------------------- 2026-02-12T18:13:11.556 INFO:teuthology.orchestra.run.trial116.stdout:Total 2.1 MB/s | 2.0 MB 00:00 2026-02-12T18:13:11.583 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction check 2026-02-12T18:13:11.590 INFO:teuthology.orchestra.run.trial116.stdout:Transaction check succeeded. 2026-02-12T18:13:11.590 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction test 2026-02-12T18:13:11.630 INFO:teuthology.orchestra.run.trial116.stdout:Transaction test succeeded. 2026-02-12T18:13:11.630 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction 2026-02-12T18:13:11.705 INFO:teuthology.orchestra.run.trial116.stdout: Preparing : 1/1 2026-02-12T18:13:11.756 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/10 2026-02-12T18:13:11.769 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/10 2026-02-12T18:13:11.783 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/10 2026-02-12T18:13:11.788 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/10 2026-02-12T18:13:11.796 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/10 2026-02-12T18:13:11.835 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/10 2026-02-12T18:13:11.847 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 7/10 2026-02-12T18:13:12.168 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 8/10 2026-02-12T18:13:12.182 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 9/10 2026-02-12T18:13:12.183 INFO:teuthology.orchestra.run.trial116.stdout: Installing : ceph-mgr-rook-2:20.3.0-5230.gafefaa40.el9.noarch 10/10 2026-02-12T18:13:12.292 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5230.gafefaa40.el9.noarch 10/10 2026-02-12T18:13:12.292 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5230.gafefaa40.el9.noarch 1/10 2026-02-12T18:13:12.292 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 2/10 2026-02-12T18:13:12.292 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 3/10 2026-02-12T18:13:12.292 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 4/10 2026-02-12T18:13:12.292 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 5/10 2026-02-12T18:13:12.292 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 6/10 2026-02-12T18:13:12.293 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 7/10 2026-02-12T18:13:12.293 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 8/10 2026-02-12T18:13:12.293 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 9/10 2026-02-12T18:13:12.350 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 10/10 2026-02-12T18:13:12.351 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:12.351 INFO:teuthology.orchestra.run.trial116.stdout:Installed: 2026-02-12T18:13:12.351 INFO:teuthology.orchestra.run.trial116.stdout: ceph-mgr-rook-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-12T18:13:12.351 INFO:teuthology.orchestra.run.trial116.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-12T18:13:12.351 INFO:teuthology.orchestra.run.trial116.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-12T18:13:12.351 INFO:teuthology.orchestra.run.trial116.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-12T18:13:12.351 INFO:teuthology.orchestra.run.trial116.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-12T18:13:12.351 INFO:teuthology.orchestra.run.trial116.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-12T18:13:12.351 INFO:teuthology.orchestra.run.trial116.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-12T18:13:12.351 INFO:teuthology.orchestra.run.trial116.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-12T18:13:12.351 INFO:teuthology.orchestra.run.trial116.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-12T18:13:12.352 INFO:teuthology.orchestra.run.trial116.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-12T18:13:12.352 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:12.352 INFO:teuthology.orchestra.run.trial116.stdout:Complete! 2026-02-12T18:13:12.383 DEBUG:teuthology.orchestra.run.trial116:> sudo yum -y install ceph-mgr-cephadm 2026-02-12T18:13:12.683 INFO:teuthology.orchestra.run.trial116.stdout:Last metadata expiration check: 0:00:46 ago on Thu 12 Feb 2026 06:12:26 PM UTC. 2026-02-12T18:13:12.764 INFO:teuthology.orchestra.run.trial116.stdout:Dependencies resolved. 2026-02-12T18:13:12.764 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:13:12.765 INFO:teuthology.orchestra.run.trial116.stdout: Package Arch Version Repository Size 2026-02-12T18:13:12.765 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:13:12.765 INFO:teuthology.orchestra.run.trial116.stdout:Installing: 2026-02-12T18:13:12.765 INFO:teuthology.orchestra.run.trial116.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 184 k 2026-02-12T18:13:12.765 INFO:teuthology.orchestra.run.trial116.stdout:Installing dependencies: 2026-02-12T18:13:12.765 INFO:teuthology.orchestra.run.trial116.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-12T18:13:12.765 INFO:teuthology.orchestra.run.trial116.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-12T18:13:12.765 INFO:teuthology.orchestra.run.trial116.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-12T18:13:12.765 INFO:teuthology.orchestra.run.trial116.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-12T18:13:12.765 INFO:teuthology.orchestra.run.trial116.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-12T18:13:12.766 INFO:teuthology.orchestra.run.trial116.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-12T18:13:12.766 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:12.766 INFO:teuthology.orchestra.run.trial116.stdout:Transaction Summary 2026-02-12T18:13:12.766 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:13:12.766 INFO:teuthology.orchestra.run.trial116.stdout:Install 7 Packages 2026-02-12T18:13:12.766 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:12.766 INFO:teuthology.orchestra.run.trial116.stdout:Total download size: 7.1 M 2026-02-12T18:13:12.766 INFO:teuthology.orchestra.run.trial116.stdout:Installed size: 34 M 2026-02-12T18:13:12.766 INFO:teuthology.orchestra.run.trial116.stdout:Downloading Packages: 2026-02-12T18:13:12.988 INFO:teuthology.orchestra.run.trial116.stdout:(1/7): ceph-mgr-cephadm-20.3.0-5230.gafefaa40.e 2.2 MB/s | 184 kB 00:00 2026-02-12T18:13:13.068 INFO:teuthology.orchestra.run.trial116.stdout:(2/7): python3-jinja2-2.11.3-8.el9.noarch.rpm 1.5 MB/s | 249 kB 00:00 2026-02-12T18:13:13.072 INFO:teuthology.orchestra.run.trial116.stdout:(3/7): python3-markupsafe-1.1.1-12.el9.x86_64.r 413 kB/s | 35 kB 00:00 2026-02-12T18:13:13.190 INFO:teuthology.orchestra.run.trial116.stdout:(4/7): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 6.0 MB 00:00 2026-02-12T18:13:13.190 INFO:teuthology.orchestra.run.trial116.stdout:(5/7): python3-natsort-7.1.1-5.el9.noarch.rpm 490 kB/s | 58 kB 00:00 2026-02-12T18:13:13.239 INFO:teuthology.orchestra.run.trial116.stdout:(6/7): python3-typing-extensions-4.15.0-1.el9.n 1.7 MB/s | 86 kB 00:00 2026-02-12T18:13:13.258 INFO:teuthology.orchestra.run.trial116.stdout:(7/7): python3-asyncssh-2.13.2-5.el9.noarch.rpm 2.8 MB/s | 548 kB 00:00 2026-02-12T18:13:13.259 INFO:teuthology.orchestra.run.trial116.stdout:-------------------------------------------------------------------------------- 2026-02-12T18:13:13.259 INFO:teuthology.orchestra.run.trial116.stdout:Total 14 MB/s | 7.1 MB 00:00 2026-02-12T18:13:13.295 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction check 2026-02-12T18:13:13.301 INFO:teuthology.orchestra.run.trial116.stdout:Transaction check succeeded. 2026-02-12T18:13:13.301 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction test 2026-02-12T18:13:13.372 INFO:teuthology.orchestra.run.trial116.stdout:Transaction test succeeded. 2026-02-12T18:13:13.373 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction 2026-02-12T18:13:13.463 INFO:teuthology.orchestra.run.trial116.stdout: Preparing : 1/1 2026-02-12T18:13:13.511 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 1/7 2026-02-12T18:13:13.521 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/7 2026-02-12T18:13:13.526 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/7 2026-02-12T18:13:13.744 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 4/7 2026-02-12T18:13:13.767 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 5/7 2026-02-12T18:13:13.791 INFO:teuthology.orchestra.run.trial116.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 6/7 2026-02-12T18:13:13.792 INFO:teuthology.orchestra.run.trial116.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 7/7 2026-02-12T18:13:13.907 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 7/7 2026-02-12T18:13:13.907 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 1/7 2026-02-12T18:13:13.908 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/7 2026-02-12T18:13:13.908 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 3/7 2026-02-12T18:13:13.908 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 4/7 2026-02-12T18:13:13.908 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 5/7 2026-02-12T18:13:13.908 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 6/7 2026-02-12T18:13:13.966 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 7/7 2026-02-12T18:13:13.966 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:13.966 INFO:teuthology.orchestra.run.trial116.stdout:Installed: 2026-02-12T18:13:13.966 INFO:teuthology.orchestra.run.trial116.stdout: ceph-mgr-cephadm-2:20.3.0-5230.gafefaa40.el9.noarch 2026-02-12T18:13:13.967 INFO:teuthology.orchestra.run.trial116.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-12T18:13:13.967 INFO:teuthology.orchestra.run.trial116.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-12T18:13:13.967 INFO:teuthology.orchestra.run.trial116.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-12T18:13:13.967 INFO:teuthology.orchestra.run.trial116.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-12T18:13:13.967 INFO:teuthology.orchestra.run.trial116.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-12T18:13:13.967 INFO:teuthology.orchestra.run.trial116.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-12T18:13:13.967 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:13.967 INFO:teuthology.orchestra.run.trial116.stdout:Complete! 2026-02-12T18:13:13.996 DEBUG:teuthology.orchestra.run.trial116:> sudo yum -y install ceph-osd 2026-02-12T18:13:14.294 INFO:teuthology.orchestra.run.trial116.stdout:Last metadata expiration check: 0:00:48 ago on Thu 12 Feb 2026 06:12:26 PM UTC. 2026-02-12T18:13:14.361 INFO:teuthology.orchestra.run.trial116.stdout:Package ceph-osd-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-12T18:13:14.379 INFO:teuthology.orchestra.run.trial116.stdout:Dependencies resolved. 2026-02-12T18:13:14.380 INFO:teuthology.orchestra.run.trial116.stdout:Nothing to do. 2026-02-12T18:13:14.380 INFO:teuthology.orchestra.run.trial116.stdout:Complete! 2026-02-12T18:13:14.401 DEBUG:teuthology.orchestra.run.trial116:> sudo yum -y install ceph-osd-classic 2026-02-12T18:13:14.706 INFO:teuthology.orchestra.run.trial116.stdout:Last metadata expiration check: 0:00:48 ago on Thu 12 Feb 2026 06:12:26 PM UTC. 2026-02-12T18:13:14.773 INFO:teuthology.orchestra.run.trial116.stdout:Package ceph-osd-classic-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-12T18:13:14.791 INFO:teuthology.orchestra.run.trial116.stdout:Dependencies resolved. 2026-02-12T18:13:14.792 INFO:teuthology.orchestra.run.trial116.stdout:Nothing to do. 2026-02-12T18:13:14.792 INFO:teuthology.orchestra.run.trial116.stdout:Complete! 2026-02-12T18:13:14.812 DEBUG:teuthology.orchestra.run.trial116:> sudo yum -y install ceph-fuse 2026-02-12T18:13:15.112 INFO:teuthology.orchestra.run.trial116.stdout:Last metadata expiration check: 0:00:49 ago on Thu 12 Feb 2026 06:12:26 PM UTC. 2026-02-12T18:13:15.196 INFO:teuthology.orchestra.run.trial116.stdout:Dependencies resolved. 2026-02-12T18:13:15.197 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:13:15.197 INFO:teuthology.orchestra.run.trial116.stdout: Package Arch Version Repo Size 2026-02-12T18:13:15.197 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:13:15.197 INFO:teuthology.orchestra.run.trial116.stdout:Installing: 2026-02-12T18:13:15.198 INFO:teuthology.orchestra.run.trial116.stdout: ceph-fuse x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 987 k 2026-02-12T18:13:15.198 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:15.198 INFO:teuthology.orchestra.run.trial116.stdout:Transaction Summary 2026-02-12T18:13:15.198 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:13:15.198 INFO:teuthology.orchestra.run.trial116.stdout:Install 1 Package 2026-02-12T18:13:15.198 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:15.198 INFO:teuthology.orchestra.run.trial116.stdout:Total download size: 987 k 2026-02-12T18:13:15.198 INFO:teuthology.orchestra.run.trial116.stdout:Installed size: 2.9 M 2026-02-12T18:13:15.198 INFO:teuthology.orchestra.run.trial116.stdout:Downloading Packages: 2026-02-12T18:13:15.316 INFO:teuthology.orchestra.run.trial116.stdout:ceph-fuse-20.3.0-5230.gafefaa40.el9.x86_64.rpm 8.1 MB/s | 987 kB 00:00 2026-02-12T18:13:15.316 INFO:teuthology.orchestra.run.trial116.stdout:-------------------------------------------------------------------------------- 2026-02-12T18:13:15.316 INFO:teuthology.orchestra.run.trial116.stdout:Total 8.1 MB/s | 987 kB 00:00 2026-02-12T18:13:15.316 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction check 2026-02-12T18:13:15.321 INFO:teuthology.orchestra.run.trial116.stdout:Transaction check succeeded. 2026-02-12T18:13:15.321 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction test 2026-02-12T18:13:15.357 INFO:teuthology.orchestra.run.trial116.stdout:Transaction test succeeded. 2026-02-12T18:13:15.358 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction 2026-02-12T18:13:15.451 INFO:teuthology.orchestra.run.trial116.stdout: Preparing : 1/1 2026-02-12T18:13:15.467 INFO:teuthology.orchestra.run.trial116.stdout: Installing : ceph-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-12T18:13:15.680 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-12T18:13:15.736 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : ceph-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-12T18:13:15.736 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:15.736 INFO:teuthology.orchestra.run.trial116.stdout:Installed: 2026-02-12T18:13:15.736 INFO:teuthology.orchestra.run.trial116.stdout: ceph-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:13:15.737 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:15.737 INFO:teuthology.orchestra.run.trial116.stdout:Complete! 2026-02-12T18:13:15.764 DEBUG:teuthology.orchestra.run.trial116:> sudo yum -y install ceph-volume 2026-02-12T18:13:16.061 INFO:teuthology.orchestra.run.trial116.stdout:Last metadata expiration check: 0:00:50 ago on Thu 12 Feb 2026 06:12:26 PM UTC. 2026-02-12T18:13:16.144 INFO:teuthology.orchestra.run.trial116.stdout:Dependencies resolved. 2026-02-12T18:13:16.145 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:13:16.145 INFO:teuthology.orchestra.run.trial116.stdout: Package Arch Version Repository Size 2026-02-12T18:13:16.145 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:13:16.145 INFO:teuthology.orchestra.run.trial116.stdout:Installing: 2026-02-12T18:13:16.146 INFO:teuthology.orchestra.run.trial116.stdout: ceph-volume noarch 2:20.3.0-5230.gafefaa40.el9 ceph-noarch 299 k 2026-02-12T18:13:16.146 INFO:teuthology.orchestra.run.trial116.stdout:Installing dependencies: 2026-02-12T18:13:16.146 INFO:teuthology.orchestra.run.trial116.stdout: cryptsetup x86_64 2.8.1-2.el9 baseos 350 k 2026-02-12T18:13:16.146 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:16.146 INFO:teuthology.orchestra.run.trial116.stdout:Transaction Summary 2026-02-12T18:13:16.146 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:13:16.146 INFO:teuthology.orchestra.run.trial116.stdout:Install 2 Packages 2026-02-12T18:13:16.146 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:16.146 INFO:teuthology.orchestra.run.trial116.stdout:Total download size: 649 k 2026-02-12T18:13:16.146 INFO:teuthology.orchestra.run.trial116.stdout:Installed size: 2.2 M 2026-02-12T18:13:16.146 INFO:teuthology.orchestra.run.trial116.stdout:Downloading Packages: 2026-02-12T18:13:16.546 INFO:teuthology.orchestra.run.trial116.stdout:(1/2): ceph-volume-20.3.0-5230.gafefaa40.el9.no 3.0 MB/s | 299 kB 00:00 2026-02-12T18:13:17.414 INFO:teuthology.orchestra.run.trial116.stdout:(2/2): cryptsetup-2.8.1-2.el9.x86_64.rpm 364 kB/s | 350 kB 00:00 2026-02-12T18:13:17.416 INFO:teuthology.orchestra.run.trial116.stdout:-------------------------------------------------------------------------------- 2026-02-12T18:13:17.416 INFO:teuthology.orchestra.run.trial116.stdout:Total 511 kB/s | 649 kB 00:01 2026-02-12T18:13:17.426 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction check 2026-02-12T18:13:17.433 INFO:teuthology.orchestra.run.trial116.stdout:Transaction check succeeded. 2026-02-12T18:13:17.433 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction test 2026-02-12T18:13:17.479 INFO:teuthology.orchestra.run.trial116.stdout:Transaction test succeeded. 2026-02-12T18:13:17.479 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction 2026-02-12T18:13:17.558 INFO:teuthology.orchestra.run.trial116.stdout: Preparing : 1/1 2026-02-12T18:13:17.621 INFO:teuthology.orchestra.run.trial116.stdout: Installing : cryptsetup-2.8.1-2.el9.x86_64 1/2 2026-02-12T18:13:17.623 INFO:teuthology.orchestra.run.trial116.stdout: Installing : ceph-volume-2:20.3.0-5230.gafefaa40.el9.noarch 2/2 2026-02-12T18:13:17.638 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: ceph-volume-2:20.3.0-5230.gafefaa40.el9.noarch 2/2 2026-02-12T18:13:17.638 INFO:teuthology.orchestra.run.trial116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T18:13:17.638 INFO:teuthology.orchestra.run.trial116.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-12T18:13:17.639 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:17.856 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : ceph-volume-2:20.3.0-5230.gafefaa40.el9.noarch 1/2 2026-02-12T18:13:17.914 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : cryptsetup-2.8.1-2.el9.x86_64 2/2 2026-02-12T18:13:17.914 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:17.914 INFO:teuthology.orchestra.run.trial116.stdout:Installed: 2026-02-12T18:13:17.914 INFO:teuthology.orchestra.run.trial116.stdout: ceph-volume-2:20.3.0-5230.gafefaa40.el9.noarch cryptsetup-2.8.1-2.el9.x86_64 2026-02-12T18:13:17.914 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:17.915 INFO:teuthology.orchestra.run.trial116.stdout:Complete! 2026-02-12T18:13:17.941 DEBUG:teuthology.orchestra.run.trial116:> sudo yum -y install librados-devel 2026-02-12T18:13:18.242 INFO:teuthology.orchestra.run.trial116.stdout:Last metadata expiration check: 0:00:52 ago on Thu 12 Feb 2026 06:12:26 PM UTC. 2026-02-12T18:13:18.326 INFO:teuthology.orchestra.run.trial116.stdout:Dependencies resolved. 2026-02-12T18:13:18.327 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:13:18.327 INFO:teuthology.orchestra.run.trial116.stdout: Package Arch Version Repo Size 2026-02-12T18:13:18.327 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:13:18.327 INFO:teuthology.orchestra.run.trial116.stdout:Installing: 2026-02-12T18:13:18.327 INFO:teuthology.orchestra.run.trial116.stdout: librados-devel x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 123 k 2026-02-12T18:13:18.327 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:18.327 INFO:teuthology.orchestra.run.trial116.stdout:Transaction Summary 2026-02-12T18:13:18.327 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:13:18.327 INFO:teuthology.orchestra.run.trial116.stdout:Install 1 Package 2026-02-12T18:13:18.328 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:18.328 INFO:teuthology.orchestra.run.trial116.stdout:Total download size: 123 k 2026-02-12T18:13:18.328 INFO:teuthology.orchestra.run.trial116.stdout:Installed size: 437 k 2026-02-12T18:13:18.328 INFO:teuthology.orchestra.run.trial116.stdout:Downloading Packages: 2026-02-12T18:13:18.412 INFO:teuthology.orchestra.run.trial116.stdout:librados-devel-20.3.0-5230.gafefaa40.el9.x86_64 1.4 MB/s | 123 kB 00:00 2026-02-12T18:13:18.412 INFO:teuthology.orchestra.run.trial116.stdout:-------------------------------------------------------------------------------- 2026-02-12T18:13:18.412 INFO:teuthology.orchestra.run.trial116.stdout:Total 1.4 MB/s | 123 kB 00:00 2026-02-12T18:13:18.412 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction check 2026-02-12T18:13:18.414 INFO:teuthology.orchestra.run.trial116.stdout:Transaction check succeeded. 2026-02-12T18:13:18.414 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction test 2026-02-12T18:13:18.446 INFO:teuthology.orchestra.run.trial116.stdout:Transaction test succeeded. 2026-02-12T18:13:18.446 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction 2026-02-12T18:13:18.499 INFO:teuthology.orchestra.run.trial116.stdout: Preparing : 1/1 2026-02-12T18:13:18.515 INFO:teuthology.orchestra.run.trial116.stdout: Installing : librados-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-12T18:13:18.596 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: librados-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-12T18:13:18.654 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : librados-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-12T18:13:18.654 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:18.654 INFO:teuthology.orchestra.run.trial116.stdout:Installed: 2026-02-12T18:13:18.654 INFO:teuthology.orchestra.run.trial116.stdout: librados-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:13:18.654 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:18.654 INFO:teuthology.orchestra.run.trial116.stdout:Complete! 2026-02-12T18:13:18.683 DEBUG:teuthology.orchestra.run.trial116:> sudo yum -y install libcephfs2 2026-02-12T18:13:18.984 INFO:teuthology.orchestra.run.trial116.stdout:Last metadata expiration check: 0:00:52 ago on Thu 12 Feb 2026 06:12:26 PM UTC. 2026-02-12T18:13:19.050 INFO:teuthology.orchestra.run.trial116.stdout:Package libcephfs2-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-12T18:13:19.068 INFO:teuthology.orchestra.run.trial116.stdout:Dependencies resolved. 2026-02-12T18:13:19.068 INFO:teuthology.orchestra.run.trial116.stdout:Nothing to do. 2026-02-12T18:13:19.069 INFO:teuthology.orchestra.run.trial116.stdout:Complete! 2026-02-12T18:13:19.089 DEBUG:teuthology.orchestra.run.trial116:> sudo yum -y install libcephfs-devel 2026-02-12T18:13:19.393 INFO:teuthology.orchestra.run.trial116.stdout:Last metadata expiration check: 0:00:53 ago on Thu 12 Feb 2026 06:12:26 PM UTC. 2026-02-12T18:13:19.477 INFO:teuthology.orchestra.run.trial116.stdout:Dependencies resolved. 2026-02-12T18:13:19.478 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:13:19.478 INFO:teuthology.orchestra.run.trial116.stdout: Package Arch Version Repo Size 2026-02-12T18:13:19.478 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:13:19.478 INFO:teuthology.orchestra.run.trial116.stdout:Installing: 2026-02-12T18:13:19.478 INFO:teuthology.orchestra.run.trial116.stdout: libcephfs-devel x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 37 k 2026-02-12T18:13:19.478 INFO:teuthology.orchestra.run.trial116.stdout:Installing dependencies: 2026-02-12T18:13:19.478 INFO:teuthology.orchestra.run.trial116.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 25 k 2026-02-12T18:13:19.478 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:19.479 INFO:teuthology.orchestra.run.trial116.stdout:Transaction Summary 2026-02-12T18:13:19.479 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:13:19.479 INFO:teuthology.orchestra.run.trial116.stdout:Install 2 Packages 2026-02-12T18:13:19.479 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:19.479 INFO:teuthology.orchestra.run.trial116.stdout:Total download size: 62 k 2026-02-12T18:13:19.479 INFO:teuthology.orchestra.run.trial116.stdout:Installed size: 223 k 2026-02-12T18:13:19.479 INFO:teuthology.orchestra.run.trial116.stdout:Downloading Packages: 2026-02-12T18:13:19.540 INFO:teuthology.orchestra.run.trial116.stdout:(1/2): libcephfs-devel-20.3.0-5230.gafefaa40.el 604 kB/s | 37 kB 00:00 2026-02-12T18:13:19.542 INFO:teuthology.orchestra.run.trial116.stdout:(2/2): libcephfs-proxy2-20.3.0-5230.gafefaa40.e 385 kB/s | 25 kB 00:00 2026-02-12T18:13:19.543 INFO:teuthology.orchestra.run.trial116.stdout:-------------------------------------------------------------------------------- 2026-02-12T18:13:19.544 INFO:teuthology.orchestra.run.trial116.stdout:Total 943 kB/s | 62 kB 00:00 2026-02-12T18:13:19.544 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction check 2026-02-12T18:13:19.546 INFO:teuthology.orchestra.run.trial116.stdout:Transaction check succeeded. 2026-02-12T18:13:19.546 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction test 2026-02-12T18:13:19.578 INFO:teuthology.orchestra.run.trial116.stdout:Transaction test succeeded. 2026-02-12T18:13:19.579 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction 2026-02-12T18:13:19.628 INFO:teuthology.orchestra.run.trial116.stdout: Preparing : 1/1 2026-02-12T18:13:19.629 INFO:teuthology.orchestra.run.trial116.stdout: Installing : libcephfs-proxy2-2:20.3.0-5230.gafefaa40.el9.x86_64 1/2 2026-02-12T18:13:19.645 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5230.gafefaa40.el9.x86_64 1/2 2026-02-12T18:13:19.661 INFO:teuthology.orchestra.run.trial116.stdout: Installing : libcephfs-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 2/2 2026-02-12T18:13:19.719 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 2/2 2026-02-12T18:13:19.719 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : libcephfs-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 1/2 2026-02-12T18:13:19.777 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5230.gafefaa40.el9.x86_64 2/2 2026-02-12T18:13:19.777 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:19.777 INFO:teuthology.orchestra.run.trial116.stdout:Installed: 2026-02-12T18:13:19.777 INFO:teuthology.orchestra.run.trial116.stdout: libcephfs-devel-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:13:19.777 INFO:teuthology.orchestra.run.trial116.stdout: libcephfs-proxy2-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:13:19.781 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:19.781 INFO:teuthology.orchestra.run.trial116.stdout:Complete! 2026-02-12T18:13:19.806 DEBUG:teuthology.orchestra.run.trial116:> sudo yum -y install librados2 2026-02-12T18:13:20.109 INFO:teuthology.orchestra.run.trial116.stdout:Last metadata expiration check: 0:00:54 ago on Thu 12 Feb 2026 06:12:26 PM UTC. 2026-02-12T18:13:20.175 INFO:teuthology.orchestra.run.trial116.stdout:Package librados2-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-12T18:13:20.193 INFO:teuthology.orchestra.run.trial116.stdout:Dependencies resolved. 2026-02-12T18:13:20.194 INFO:teuthology.orchestra.run.trial116.stdout:Nothing to do. 2026-02-12T18:13:20.194 INFO:teuthology.orchestra.run.trial116.stdout:Complete! 2026-02-12T18:13:20.216 DEBUG:teuthology.orchestra.run.trial116:> sudo yum -y install librbd1 2026-02-12T18:13:20.510 INFO:teuthology.orchestra.run.trial116.stdout:Last metadata expiration check: 0:00:54 ago on Thu 12 Feb 2026 06:12:26 PM UTC. 2026-02-12T18:13:20.574 INFO:teuthology.orchestra.run.trial116.stdout:Package librbd1-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-12T18:13:20.593 INFO:teuthology.orchestra.run.trial116.stdout:Dependencies resolved. 2026-02-12T18:13:20.593 INFO:teuthology.orchestra.run.trial116.stdout:Nothing to do. 2026-02-12T18:13:20.593 INFO:teuthology.orchestra.run.trial116.stdout:Complete! 2026-02-12T18:13:20.615 DEBUG:teuthology.orchestra.run.trial116:> sudo yum -y install python3-rados 2026-02-12T18:13:20.918 INFO:teuthology.orchestra.run.trial116.stdout:Last metadata expiration check: 0:00:54 ago on Thu 12 Feb 2026 06:12:26 PM UTC. 2026-02-12T18:13:20.986 INFO:teuthology.orchestra.run.trial116.stdout:Package python3-rados-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-12T18:13:21.004 INFO:teuthology.orchestra.run.trial116.stdout:Dependencies resolved. 2026-02-12T18:13:21.005 INFO:teuthology.orchestra.run.trial116.stdout:Nothing to do. 2026-02-12T18:13:21.005 INFO:teuthology.orchestra.run.trial116.stdout:Complete! 2026-02-12T18:13:21.026 DEBUG:teuthology.orchestra.run.trial116:> sudo yum -y install python3-rgw 2026-02-12T18:13:21.324 INFO:teuthology.orchestra.run.trial116.stdout:Last metadata expiration check: 0:00:55 ago on Thu 12 Feb 2026 06:12:26 PM UTC. 2026-02-12T18:13:21.390 INFO:teuthology.orchestra.run.trial116.stdout:Package python3-rgw-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-12T18:13:21.408 INFO:teuthology.orchestra.run.trial116.stdout:Dependencies resolved. 2026-02-12T18:13:21.409 INFO:teuthology.orchestra.run.trial116.stdout:Nothing to do. 2026-02-12T18:13:21.410 INFO:teuthology.orchestra.run.trial116.stdout:Complete! 2026-02-12T18:13:21.430 DEBUG:teuthology.orchestra.run.trial116:> sudo yum -y install python3-cephfs 2026-02-12T18:13:21.728 INFO:teuthology.orchestra.run.trial116.stdout:Last metadata expiration check: 0:00:55 ago on Thu 12 Feb 2026 06:12:26 PM UTC. 2026-02-12T18:13:21.792 INFO:teuthology.orchestra.run.trial116.stdout:Package python3-cephfs-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-12T18:13:21.810 INFO:teuthology.orchestra.run.trial116.stdout:Dependencies resolved. 2026-02-12T18:13:21.811 INFO:teuthology.orchestra.run.trial116.stdout:Nothing to do. 2026-02-12T18:13:21.811 INFO:teuthology.orchestra.run.trial116.stdout:Complete! 2026-02-12T18:13:21.832 DEBUG:teuthology.orchestra.run.trial116:> sudo yum -y install python3-rbd 2026-02-12T18:13:22.136 INFO:teuthology.orchestra.run.trial116.stdout:Last metadata expiration check: 0:00:56 ago on Thu 12 Feb 2026 06:12:26 PM UTC. 2026-02-12T18:13:22.202 INFO:teuthology.orchestra.run.trial116.stdout:Package python3-rbd-2:20.3.0-5230.gafefaa40.el9.x86_64 is already installed. 2026-02-12T18:13:22.220 INFO:teuthology.orchestra.run.trial116.stdout:Dependencies resolved. 2026-02-12T18:13:22.220 INFO:teuthology.orchestra.run.trial116.stdout:Nothing to do. 2026-02-12T18:13:22.221 INFO:teuthology.orchestra.run.trial116.stdout:Complete! 2026-02-12T18:13:22.242 DEBUG:teuthology.orchestra.run.trial116:> sudo yum -y install rbd-fuse 2026-02-12T18:13:22.544 INFO:teuthology.orchestra.run.trial116.stdout:Last metadata expiration check: 0:00:56 ago on Thu 12 Feb 2026 06:12:26 PM UTC. 2026-02-12T18:13:22.628 INFO:teuthology.orchestra.run.trial116.stdout:Dependencies resolved. 2026-02-12T18:13:22.629 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:13:22.629 INFO:teuthology.orchestra.run.trial116.stdout: Package Architecture Version Repository Size 2026-02-12T18:13:22.629 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:13:22.629 INFO:teuthology.orchestra.run.trial116.stdout:Installing: 2026-02-12T18:13:22.629 INFO:teuthology.orchestra.run.trial116.stdout: rbd-fuse x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 84 k 2026-02-12T18:13:22.629 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:22.629 INFO:teuthology.orchestra.run.trial116.stdout:Transaction Summary 2026-02-12T18:13:22.630 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:13:22.630 INFO:teuthology.orchestra.run.trial116.stdout:Install 1 Package 2026-02-12T18:13:22.630 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:22.630 INFO:teuthology.orchestra.run.trial116.stdout:Total download size: 84 k 2026-02-12T18:13:22.630 INFO:teuthology.orchestra.run.trial116.stdout:Installed size: 209 k 2026-02-12T18:13:22.630 INFO:teuthology.orchestra.run.trial116.stdout:Downloading Packages: 2026-02-12T18:13:22.714 INFO:teuthology.orchestra.run.trial116.stdout:rbd-fuse-20.3.0-5230.gafefaa40.el9.x86_64.rpm 997 kB/s | 84 kB 00:00 2026-02-12T18:13:22.714 INFO:teuthology.orchestra.run.trial116.stdout:-------------------------------------------------------------------------------- 2026-02-12T18:13:22.715 INFO:teuthology.orchestra.run.trial116.stdout:Total 986 kB/s | 84 kB 00:00 2026-02-12T18:13:22.715 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction check 2026-02-12T18:13:22.718 INFO:teuthology.orchestra.run.trial116.stdout:Transaction check succeeded. 2026-02-12T18:13:22.718 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction test 2026-02-12T18:13:22.749 INFO:teuthology.orchestra.run.trial116.stdout:Transaction test succeeded. 2026-02-12T18:13:22.749 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction 2026-02-12T18:13:22.801 INFO:teuthology.orchestra.run.trial116.stdout: Preparing : 1/1 2026-02-12T18:13:22.817 INFO:teuthology.orchestra.run.trial116.stdout: Installing : rbd-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-12T18:13:22.893 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-12T18:13:22.951 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : rbd-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-12T18:13:22.952 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:22.952 INFO:teuthology.orchestra.run.trial116.stdout:Installed: 2026-02-12T18:13:22.952 INFO:teuthology.orchestra.run.trial116.stdout: rbd-fuse-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:13:22.952 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:22.952 INFO:teuthology.orchestra.run.trial116.stdout:Complete! 2026-02-12T18:13:22.980 DEBUG:teuthology.orchestra.run.trial116:> sudo yum -y install rbd-mirror 2026-02-12T18:13:23.282 INFO:teuthology.orchestra.run.trial116.stdout:Last metadata expiration check: 0:00:57 ago on Thu 12 Feb 2026 06:12:26 PM UTC. 2026-02-12T18:13:23.365 INFO:teuthology.orchestra.run.trial116.stdout:Dependencies resolved. 2026-02-12T18:13:23.365 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:13:23.365 INFO:teuthology.orchestra.run.trial116.stdout: Package Arch Version Repo Size 2026-02-12T18:13:23.365 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:13:23.366 INFO:teuthology.orchestra.run.trial116.stdout:Installing: 2026-02-12T18:13:23.366 INFO:teuthology.orchestra.run.trial116.stdout: rbd-mirror x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 2.9 M 2026-02-12T18:13:23.366 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:23.366 INFO:teuthology.orchestra.run.trial116.stdout:Transaction Summary 2026-02-12T18:13:23.366 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:13:23.366 INFO:teuthology.orchestra.run.trial116.stdout:Install 1 Package 2026-02-12T18:13:23.366 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:23.366 INFO:teuthology.orchestra.run.trial116.stdout:Total download size: 2.9 M 2026-02-12T18:13:23.366 INFO:teuthology.orchestra.run.trial116.stdout:Installed size: 11 M 2026-02-12T18:13:23.366 INFO:teuthology.orchestra.run.trial116.stdout:Downloading Packages: 2026-02-12T18:13:23.512 INFO:teuthology.orchestra.run.trial116.stdout:rbd-mirror-20.3.0-5230.gafefaa40.el9.x86_64.rpm 20 MB/s | 2.9 MB 00:00 2026-02-12T18:13:23.512 INFO:teuthology.orchestra.run.trial116.stdout:-------------------------------------------------------------------------------- 2026-02-12T18:13:23.512 INFO:teuthology.orchestra.run.trial116.stdout:Total 20 MB/s | 2.9 MB 00:00 2026-02-12T18:13:23.512 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction check 2026-02-12T18:13:23.517 INFO:teuthology.orchestra.run.trial116.stdout:Transaction check succeeded. 2026-02-12T18:13:23.517 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction test 2026-02-12T18:13:23.557 INFO:teuthology.orchestra.run.trial116.stdout:Transaction test succeeded. 2026-02-12T18:13:23.557 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction 2026-02-12T18:13:23.740 INFO:teuthology.orchestra.run.trial116.stdout: Preparing : 1/1 2026-02-12T18:13:23.742 INFO:teuthology.orchestra.run.trial116.stdout: Installing : rbd-mirror-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-12T18:13:23.764 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-12T18:13:23.764 INFO:teuthology.orchestra.run.trial116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T18:13:23.764 INFO:teuthology.orchestra.run.trial116.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-12T18:13:23.764 INFO:teuthology.orchestra.run.trial116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-12T18:13:23.764 INFO:teuthology.orchestra.run.trial116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-12T18:13:23.764 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:24.086 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : rbd-mirror-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-12T18:13:24.086 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:24.086 INFO:teuthology.orchestra.run.trial116.stdout:Installed: 2026-02-12T18:13:24.086 INFO:teuthology.orchestra.run.trial116.stdout: rbd-mirror-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:13:24.087 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:24.087 INFO:teuthology.orchestra.run.trial116.stdout:Complete! 2026-02-12T18:13:24.113 DEBUG:teuthology.orchestra.run.trial116:> sudo yum -y install rbd-nbd 2026-02-12T18:13:24.414 INFO:teuthology.orchestra.run.trial116.stdout:Last metadata expiration check: 0:00:58 ago on Thu 12 Feb 2026 06:12:26 PM UTC. 2026-02-12T18:13:24.499 INFO:teuthology.orchestra.run.trial116.stdout:Dependencies resolved. 2026-02-12T18:13:24.499 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:13:24.499 INFO:teuthology.orchestra.run.trial116.stdout: Package Architecture Version Repository Size 2026-02-12T18:13:24.499 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:13:24.499 INFO:teuthology.orchestra.run.trial116.stdout:Installing: 2026-02-12T18:13:24.499 INFO:teuthology.orchestra.run.trial116.stdout: rbd-nbd x86_64 2:20.3.0-5230.gafefaa40.el9 ceph 172 k 2026-02-12T18:13:24.499 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:24.500 INFO:teuthology.orchestra.run.trial116.stdout:Transaction Summary 2026-02-12T18:13:24.500 INFO:teuthology.orchestra.run.trial116.stdout:================================================================================ 2026-02-12T18:13:24.500 INFO:teuthology.orchestra.run.trial116.stdout:Install 1 Package 2026-02-12T18:13:24.500 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:24.500 INFO:teuthology.orchestra.run.trial116.stdout:Total download size: 172 k 2026-02-12T18:13:24.500 INFO:teuthology.orchestra.run.trial116.stdout:Installed size: 462 k 2026-02-12T18:13:24.500 INFO:teuthology.orchestra.run.trial116.stdout:Downloading Packages: 2026-02-12T18:13:24.584 INFO:teuthology.orchestra.run.trial116.stdout:rbd-nbd-20.3.0-5230.gafefaa40.el9.x86_64.rpm 2.0 MB/s | 172 kB 00:00 2026-02-12T18:13:24.585 INFO:teuthology.orchestra.run.trial116.stdout:-------------------------------------------------------------------------------- 2026-02-12T18:13:24.585 INFO:teuthology.orchestra.run.trial116.stdout:Total 2.0 MB/s | 172 kB 00:00 2026-02-12T18:13:24.585 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction check 2026-02-12T18:13:24.589 INFO:teuthology.orchestra.run.trial116.stdout:Transaction check succeeded. 2026-02-12T18:13:24.589 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction test 2026-02-12T18:13:24.621 INFO:teuthology.orchestra.run.trial116.stdout:Transaction test succeeded. 2026-02-12T18:13:24.621 INFO:teuthology.orchestra.run.trial116.stdout:Running transaction 2026-02-12T18:13:24.676 INFO:teuthology.orchestra.run.trial116.stdout: Preparing : 1/1 2026-02-12T18:13:24.693 INFO:teuthology.orchestra.run.trial116.stdout: Installing : rbd-nbd-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-12T18:13:24.774 INFO:teuthology.orchestra.run.trial116.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-12T18:13:24.836 INFO:teuthology.orchestra.run.trial116.stdout: Verifying : rbd-nbd-2:20.3.0-5230.gafefaa40.el9.x86_64 1/1 2026-02-12T18:13:24.836 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:24.836 INFO:teuthology.orchestra.run.trial116.stdout:Installed: 2026-02-12T18:13:24.836 INFO:teuthology.orchestra.run.trial116.stdout: rbd-nbd-2:20.3.0-5230.gafefaa40.el9.x86_64 2026-02-12T18:13:24.836 INFO:teuthology.orchestra.run.trial116.stdout: 2026-02-12T18:13:24.836 INFO:teuthology.orchestra.run.trial116.stdout:Complete! 2026-02-12T18:13:24.868 DEBUG:teuthology.parallel:result is None 2026-02-12T18:13:24.869 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=afefaa40503a4bf024e96f97dd7f852b75f00ad6 2026-02-12T18:13:25.032 DEBUG:teuthology.orchestra.run.trial116:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-12T18:13:25.053 INFO:teuthology.orchestra.run.trial116.stdout:20.3.0-5230.gafefaa40.el9 2026-02-12T18:13:25.053 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5230.gafefaa40.el9 2026-02-12T18:13:25.053 INFO:teuthology.task.install:The correct ceph version 20.3.0-5230.gafefaa40 is installed. 2026-02-12T18:13:25.055 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-12T18:13:25.056 DEBUG:teuthology.orchestra.run.trial116:> set -ex 2026-02-12T18:13:25.056 DEBUG:teuthology.orchestra.run.trial116:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-12T18:13:25.117 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-12T18:13:25.117 DEBUG:teuthology.orchestra.run.trial116:> set -ex 2026-02-12T18:13:25.118 DEBUG:teuthology.orchestra.run.trial116:> sudo dd of=/usr/bin/daemon-helper 2026-02-12T18:13:25.181 DEBUG:teuthology.orchestra.run.trial116:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-12T18:13:25.247 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-12T18:13:25.247 DEBUG:teuthology.orchestra.run.trial116:> set -ex 2026-02-12T18:13:25.247 DEBUG:teuthology.orchestra.run.trial116:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-12T18:13:25.311 DEBUG:teuthology.orchestra.run.trial116:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-12T18:13:25.376 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-12T18:13:25.376 DEBUG:teuthology.orchestra.run.trial116:> set -ex 2026-02-12T18:13:25.376 DEBUG:teuthology.orchestra.run.trial116:> sudo dd of=/usr/bin/stdin-killer 2026-02-12T18:13:25.440 DEBUG:teuthology.orchestra.run.trial116:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-12T18:13:25.504 INFO:teuthology.run_tasks:Running task workunit... 2026-02-12T18:13:25.532 INFO:tasks.workunit:Pulling workunits from ref afefaa40503a4bf024e96f97dd7f852b75f00ad6 2026-02-12T18:13:25.533 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-12T18:13:25.533 INFO:tasks.workunit:timeout=3h 2026-02-12T18:13:25.533 INFO:tasks.workunit:cleanup=True 2026-02-12T18:13:25.533 DEBUG:teuthology.orchestra.run.trial116:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-12T18:13:25.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-12T18:13:25.560 INFO:teuthology.orchestra.run.trial116.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-12T18:13:25.561 DEBUG:teuthology.orchestra.run.trial116:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-12T18:13:25.618 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-12T18:13:25.619 DEBUG:teuthology.orchestra.run.trial116:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-12T18:13:25.676 DEBUG:teuthology.orchestra.run.trial116:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout afefaa40503a4bf024e96f97dd7f852b75f00ad6 2026-02-12T18:13:25.738 INFO:tasks.workunit.client.0.trial116.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-12T18:13:45.171 INFO:tasks.workunit.client.0.trial116.stderr:Note: switching to 'afefaa40503a4bf024e96f97dd7f852b75f00ad6'. 2026-02-12T18:13:45.171 INFO:tasks.workunit.client.0.trial116.stderr: 2026-02-12T18:13:45.172 INFO:tasks.workunit.client.0.trial116.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-12T18:13:45.172 INFO:tasks.workunit.client.0.trial116.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-12T18:13:45.172 INFO:tasks.workunit.client.0.trial116.stderr:state without impacting any branches by switching back to a branch. 2026-02-12T18:13:45.172 INFO:tasks.workunit.client.0.trial116.stderr: 2026-02-12T18:13:45.172 INFO:tasks.workunit.client.0.trial116.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-12T18:13:45.172 INFO:tasks.workunit.client.0.trial116.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-12T18:13:45.172 INFO:tasks.workunit.client.0.trial116.stderr: 2026-02-12T18:13:45.172 INFO:tasks.workunit.client.0.trial116.stderr: git switch -c 2026-02-12T18:13:45.172 INFO:tasks.workunit.client.0.trial116.stderr: 2026-02-12T18:13:45.172 INFO:tasks.workunit.client.0.trial116.stderr:Or undo this operation with: 2026-02-12T18:13:45.172 INFO:tasks.workunit.client.0.trial116.stderr: 2026-02-12T18:13:45.173 INFO:tasks.workunit.client.0.trial116.stderr: git switch - 2026-02-12T18:13:45.173 INFO:tasks.workunit.client.0.trial116.stderr: 2026-02-12T18:13:45.173 INFO:tasks.workunit.client.0.trial116.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-12T18:13:45.173 INFO:tasks.workunit.client.0.trial116.stderr: 2026-02-12T18:13:45.173 INFO:tasks.workunit.client.0.trial116.stderr:HEAD is now at afefaa40503 Merge branch 'block_non_4k_chunks' of https://github.com/aainscow/ceph into wip-yuri4-testing-2026-02-11-2126 2026-02-12T18:13:45.178 DEBUG:teuthology.orchestra.run.trial116:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/standalone && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-12T18:13:45.238 DEBUG:teuthology.orchestra.run.trial116:> set -ex 2026-02-12T18:13:45.239 DEBUG:teuthology.orchestra.run.trial116:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-12T18:13:45.297 INFO:tasks.workunit:Running workunits matching erasure-code on client.0... 2026-02-12T18:13:45.298 INFO:tasks.workunit:Running workunit erasure-code/test-erasure-code-plugins.sh... 2026-02-12T18:13:45.298 DEBUG:teuthology.orchestra.run.trial116:workunit test erasure-code/test-erasure-code-plugins.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=afefaa40503a4bf024e96f97dd7f852b75f00ad6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh 2026-02-12T18:13:45.362 INFO:tasks.workunit.client.0.trial116.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh 2026-02-12T18:13:45.362 INFO:tasks.workunit.client.0.trial116.stderr:++ TIMEOUT=300 2026-02-12T18:13:45.362 INFO:tasks.workunit.client.0.trial116.stderr:++ WAIT_FOR_CLEAN_TIMEOUT=90 2026-02-12T18:13:45.362 INFO:tasks.workunit.client.0.trial116.stderr:++ MAX_TIMEOUT=15 2026-02-12T18:13:45.362 INFO:tasks.workunit.client.0.trial116.stderr:++ PG_NUM=4 2026-02-12T18:13:45.363 INFO:tasks.workunit.client.0.trial116.stderr:++ TMPDIR=/tmp 2026-02-12T18:13:45.363 INFO:tasks.workunit.client.0.trial116.stderr:++ CEPH_BUILD_VIRTUALENV=/tmp 2026-02-12T18:13:45.363 INFO:tasks.workunit.client.0.trial116.stderr:++ TESTDIR=/home/ubuntu/cephtest 2026-02-12T18:13:45.363 INFO:tasks.workunit.client.0.trial116.stderr:+++ uname 2026-02-12T18:13:45.364 INFO:tasks.workunit.client.0.trial116.stderr:++ '[' Linux = FreeBSD ']' 2026-02-12T18:13:45.364 INFO:tasks.workunit.client.0.trial116.stderr:++ SED=sed 2026-02-12T18:13:45.365 INFO:tasks.workunit.client.0.trial116.stderr:++ AWK=awk 2026-02-12T18:13:45.365 INFO:tasks.workunit.client.0.trial116.stderr:+++ stty -a 2026-02-12T18:13:45.365 INFO:tasks.workunit.client.0.trial116.stderr:+++ head -1 2026-02-12T18:13:45.365 INFO:tasks.workunit.client.0.trial116.stderr:+++ sed -e 's/.*columns \([0-9]*\).*/\1/' 2026-02-12T18:13:45.365 INFO:tasks.workunit.client.0.trial116.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-12T18:13:45.365 INFO:tasks.workunit.client.0.trial116.stderr:++ termwidth= 2026-02-12T18:13:45.365 INFO:tasks.workunit.client.0.trial116.stderr:++ '[' -n '' -a '' '!=' 0 ']' 2026-02-12T18:13:45.365 INFO:tasks.workunit.client.0.trial116.stderr:++ DIFFCOLOPTS='-y ' 2026-02-12T18:13:45.365 INFO:tasks.workunit.client.0.trial116.stderr:++ KERNCORE=kernel.core_pattern 2026-02-12T18:13:45.365 INFO:tasks.workunit.client.0.trial116.stderr:++ EXTRA_OPTS= 2026-02-12T18:13:45.366 INFO:tasks.workunit.client.0.trial116.stderr:++ test '' = TESTS 2026-02-12T18:13:45.366 INFO:tasks.workunit.client.0.trial116.stderr:++ uname -m 2026-02-12T18:13:45.367 INFO:tasks.workunit.client.0.trial116.stderr:+ arch=x86_64 2026-02-12T18:13:45.367 INFO:tasks.workunit.client.0.trial116.stderr:+ case $arch in 2026-02-12T18:13:45.367 INFO:tasks.workunit.client.0.trial116.stderr:+ legacy_jerasure_plugins=(jerasure_generic jerasure_sse3 jerasure_sse4) 2026-02-12T18:13:45.367 INFO:tasks.workunit.client.0.trial116.stderr:+ legacy_shec_plugins=(shec_generic shec_sse3 shec_sse4) 2026-02-12T18:13:45.367 INFO:tasks.workunit.client.0.trial116.stderr:+ plugins=(jerasure shec lrc isa) 2026-02-12T18:13:45.368 INFO:tasks.workunit.client.0.trial116.stderr:+ main test-erasure-code-plugins 2026-02-12T18:13:45.368 INFO:tasks.workunit.client.0.trial116.stderr:+ local dir=td/test-erasure-code-plugins 2026-02-12T18:13:45.368 INFO:tasks.workunit.client.0.trial116.stderr:+ shift 2026-02-12T18:13:45.368 INFO:tasks.workunit.client.0.trial116.stderr:+ shopt -s -o xtrace 2026-02-12T18:13:45.368 INFO:tasks.workunit.client.0.trial116.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-12T18:13:45.368 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-12T18:13:45.368 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-12T18:13:45.368 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-12T18:13:45.368 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-12T18:13:45.368 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-12T18:13:45.368 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-12T18:13:45.368 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-12T18:13:45.368 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-12T18:13:45.369 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/test-erasure-code-plugins 2026-02-12T18:13:45.369 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:26: run: local dir=td/test-erasure-code-plugins 2026-02-12T18:13:45.369 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:27: run: shift 2026-02-12T18:13:45.369 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:29: run: export CEPH_MON=127.0.0.1:17110 2026-02-12T18:13:45.369 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:29: run: CEPH_MON=127.0.0.1:17110 2026-02-12T18:13:45.369 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:30: run: export CEPH_ARGS 2026-02-12T18:13:45.369 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:31: run: uuidgen 2026-02-12T18:13:45.369 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:31: run: CEPH_ARGS+='--fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none ' 2026-02-12T18:13:45.369 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:32: run: CEPH_ARGS+='--mon-host=127.0.0.1:17110 ' 2026-02-12T18:13:45.369 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:34: run: set 2026-02-12T18:13:45.369 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:34: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-12T18:13:45.372 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:34: run: local 'funcs=TEST_ec_profile_blaum_roth_warning 2026-02-12T18:13:45.373 INFO:tasks.workunit.client.0.trial116.stderr:TEST_ec_profile_warning 2026-02-12T18:13:45.373 INFO:tasks.workunit.client.0.trial116.stderr:TEST_preload_no_warning 2026-02-12T18:13:45.373 INFO:tasks.workunit.client.0.trial116.stderr:TEST_preload_no_warning_default 2026-02-12T18:13:45.373 INFO:tasks.workunit.client.0.trial116.stderr:TEST_preload_warning' 2026-02-12T18:13:45.373 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-12T18:13:45.373 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_ec_profile_blaum_roth_warning td/test-erasure-code-plugins 2026-02-12T18:13:45.373 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:119: TEST_ec_profile_blaum_roth_warning: local dir=td/test-erasure-code-plugins 2026-02-12T18:13:45.373 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:121: TEST_ec_profile_blaum_roth_warning: setup td/test-erasure-code-plugins 2026-02-12T18:13:45.374 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-12T18:13:45.374 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-12T18:13:45.374 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-12T18:13:45.374 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:13:45.374 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-12T18:13:45.374 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:13:45.374 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:13:45.374 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:13:45.375 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:13:45.375 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:13:45.376 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:13:45.376 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:13:45.376 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:13:45.377 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:13:45.377 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:13:45.377 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:13:45.377 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:13:45.377 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:13:45.377 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:13:45.377 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:13:45.377 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:13:45.378 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:13:45.378 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:13:45.378 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-12T18:13:45.379 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:13:45.379 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:13:45.379 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:13:45.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19237 2026-02-12T18:13:45.380 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:13:45.380 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:13:45.380 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-12T18:13:45.381 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T18:13:45.381 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:13:45.381 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:13:45.382 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19237 2026-02-12T18:13:45.382 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T18:13:45.383 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-12T18:13:45.384 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-12T18:13:45.384 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T18:13:45.384 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-12T18:13:45.384 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:122: TEST_ec_profile_blaum_roth_warning: run_mon td/test-erasure-code-plugins a 2026-02-12T18:13:45.384 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-12T18:13:45.384 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T18:13:45.384 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T18:13:45.384 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T18:13:45.384 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-12T18:13:45.384 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins 2026-02-12T18:13:45.413 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T18:13:45.413 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:13:45.413 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:13:45.414 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:13:45.414 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:13:45.414 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:13:45.414 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:13:45.414 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T18:13:45.436 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T18:13:45.437 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T18:13:45.437 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:13:45.438 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:13:45.438 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T18:13:45.438 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T18:13:45.438 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:13:45.438 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:13:45.438 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:13:45.439 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:13:45.439 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:13:45.439 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:13:45.439 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:13:45.439 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:13:45.439 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19237/ceph-mon.a.asok config get fsid 2026-02-12T18:13:45.480 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T18:13:45.481 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:13:45.481 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:13:45.481 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T18:13:45.481 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T18:13:45.481 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:13:45.481 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:13:45.481 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:13:45.481 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:13:45.481 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:13:45.481 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:13:45.481 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:13:45.482 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:13:45.482 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19237/ceph-mon.a.asok config get mon_host 2026-02-12T18:13:45.520 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:123: TEST_ec_profile_blaum_roth_warning: run_mgr td/test-erasure-code-plugins x 2026-02-12T18:13:45.520 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-12T18:13:45.521 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T18:13:45.521 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T18:13:45.521 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T18:13:45.521 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-12T18:13:45.521 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T18:13:45.619 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T18:13:45.619 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:13:45.619 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:13:45.619 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:13:45.619 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:13:45.619 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:13:45.619 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:13:45.620 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:13:45.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:13:45.634 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:124: TEST_ec_profile_blaum_roth_warning: seq 0 2 2026-02-12T18:13:45.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:124: TEST_ec_profile_blaum_roth_warning: for id in $(seq 0 2) 2026-02-12T18:13:45.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:125: TEST_ec_profile_blaum_roth_warning: run_osd td/test-erasure-code-plugins 0 2026-02-12T18:13:45.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-12T18:13:45.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:13:45.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T18:13:45.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:13:45.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-12T18:13:45.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-12T18:13:45.639 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:13:45.639 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:13:45.639 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:13:45.639 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-12T18:13:45.639 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-12T18:13:45.639 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:13:45.639 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:13:45.639 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-12T18:13:45.639 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:13:45.639 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:13:45.639 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:13:45.639 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:13:45.640 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:13:45.640 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:13:45.640 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:13:45.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:13:45.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:13:45.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:13:45.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:13:45.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-12T18:13:45.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-12T18:13:45.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:13:45.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:13:45.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:13:45.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:13:45.641 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:13:45.641 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:13:45.641 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-12T18:13:45.641 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:13:45.641 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=35f4b82d-a438-4cb3-8ce4-acefa118e1d4 2026-02-12T18:13:45.641 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 35f4b82d-a438-4cb3-8ce4-acefa118e1d4' 2026-02-12T18:13:45.641 INFO:tasks.workunit.client.0.trial116.stdout:add osd0 35f4b82d-a438-4cb3-8ce4-acefa118e1d4 2026-02-12T18:13:45.642 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:13:45.646 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBZGI5p225CJhAA2SQOqb4Bk6G5Kc1E1Wokow== 2026-02-12T18:13:45.646 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBZGI5p225CJhAA2SQOqb4Bk6G5Kc1E1Wokow=="}' 2026-02-12T18:13:45.646 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 35f4b82d-a438-4cb3-8ce4-acefa118e1d4 -i td/test-erasure-code-plugins/0/new.json 2026-02-12T18:13:45.746 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:13:45.752 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-12T18:13:45.753 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBZGI5p225CJhAA2SQOqb4Bk6G5Kc1E1Wokow== --osd-uuid 35f4b82d-a438-4cb3-8ce4-acefa118e1d4 2026-02-12T18:13:45.768 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:13:45.763+0000 7fb732427780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:13:45.769 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:13:45.764+0000 7fb732427780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:13:45.770 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:13:45.765+0000 7fb732427780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:13:45.771 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:13:45.766+0000 7fb732427780 -1 bdev(0x3790c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-12T18:13:45.771 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:13:45.766+0000 7fb732427780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-12T18:13:46.148 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-12T18:13:46.149 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:13:46.149 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T18:13:46.149 INFO:tasks.workunit.client.0.trial116.stdout:adding osd0 key to auth repository 2026-02-12T18:13:46.150 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:13:46.250 INFO:tasks.workunit.client.0.trial116.stdout:start osd.0 2026-02-12T18:13:46.250 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T18:13:46.250 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:13:46.251 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:13:46.251 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:13:46.251 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:13:46.264 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:13:46.259+0000 7f773f012780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:13:46.265 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:13:46.261+0000 7f773f012780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:13:46.266 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:13:46.261+0000 7f773f012780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:13:46.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T18:13:46.347 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:13:46.347 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T18:13:46.347 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:13:46.347 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:13:46.347 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:13:46.347 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:13:46.347 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:13:46.348 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:13:46.348 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:13:46.440 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:13:46.462 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:13:46.457+0000 7f773f012780 -1 Falling back to public interface 2026-02-12T18:13:46.772 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:13:46.767+0000 7f773f012780 -1 osd.0 0 log_to_monitors true 2026-02-12T18:13:47.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:13:47.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:13:47.442 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:13:47.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:13:47.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:13:47.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:13:47.538 INFO:tasks.workunit.client.0.trial116.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/383157394,v1:127.0.0.1:6801/383157394] [v2:127.0.0.1:6802/383157394,v1:127.0.0.1:6803/383157394] exists,up 35f4b82d-a438-4cb3-8ce4-acefa118e1d4 2026-02-12T18:13:47.542 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:13:47.542 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:13:47.542 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:13:47.542 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:124: TEST_ec_profile_blaum_roth_warning: for id in $(seq 0 2) 2026-02-12T18:13:47.542 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:125: TEST_ec_profile_blaum_roth_warning: run_osd td/test-erasure-code-plugins 1 2026-02-12T18:13:47.542 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-12T18:13:47.542 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:13:47.542 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-12T18:13:47.542 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:13:47.542 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/1 2026-02-12T18:13:47.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-12T18:13:47.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:13:47.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:13:47.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:13:47.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/1' 2026-02-12T18:13:47.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/1/journal' 2026-02-12T18:13:47.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:13:47.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:13:47.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-12T18:13:47.543 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:13:47.543 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:13:47.543 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:13:47.544 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:13:47.544 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:13:47.544 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:13:47.544 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:13:47.544 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:13:47.544 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:13:47.544 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:13:47.544 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:13:47.544 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-12T18:13:47.544 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-12T18:13:47.544 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:13:47.544 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:13:47.544 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:13:47.545 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:13:47.545 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:13:47.545 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:13:47.545 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/1 2026-02-12T18:13:47.545 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:13:47.545 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4e37df1a-c5f7-4c64-be85-b14408b1458f 2026-02-12T18:13:47.545 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 4e37df1a-c5f7-4c64-be85-b14408b1458f' 2026-02-12T18:13:47.545 INFO:tasks.workunit.client.0.trial116.stdout:add osd1 4e37df1a-c5f7-4c64-be85-b14408b1458f 2026-02-12T18:13:47.546 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:13:47.550 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBbGI5pbfaSIBAAk+L1+xBss/eZJluOGdeOiw== 2026-02-12T18:13:47.550 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBbGI5pbfaSIBAAk+L1+xBss/eZJluOGdeOiw=="}' 2026-02-12T18:13:47.551 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4e37df1a-c5f7-4c64-be85-b14408b1458f -i td/test-erasure-code-plugins/1/new.json 2026-02-12T18:13:47.652 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:13:47.658 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/1/new.json 2026-02-12T18:13:47.659 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/1 --osd-journal=td/test-erasure-code-plugins/1/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBbGI5pbfaSIBAAk+L1+xBss/eZJluOGdeOiw== --osd-uuid 4e37df1a-c5f7-4c64-be85-b14408b1458f 2026-02-12T18:13:47.673 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:13:47.669+0000 7f8565e55780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:13:47.675 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:13:47.670+0000 7f8565e55780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:13:47.676 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:13:47.671+0000 7f8565e55780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:13:47.676 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:13:47.671+0000 7f8565e55780 -1 bdev(0x3120c00 td/test-erasure-code-plugins/1/block) open stat got: (1) Operation not permitted 2026-02-12T18:13:47.676 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:13:47.671+0000 7f8565e55780 -1 bluestore(td/test-erasure-code-plugins/1) _read_fsid unparsable uuid 2026-02-12T18:13:48.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/1/keyring 2026-02-12T18:13:48.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:13:48.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-12T18:13:48.062 INFO:tasks.workunit.client.0.trial116.stdout:adding osd1 key to auth repository 2026-02-12T18:13:48.062 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:13:48.386 INFO:tasks.workunit.client.0.trial116.stdout:start osd.1 2026-02-12T18:13:48.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-12T18:13:48.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/1 --osd-journal=td/test-erasure-code-plugins/1/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:13:48.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:13:48.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:13:48.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:13:48.400 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:13:48.395+0000 7fa753e51780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:13:48.401 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:13:48.397+0000 7fa753e51780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:13:48.402 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:13:48.397+0000 7fa753e51780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:13:48.525 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:13:48.521+0000 7fa753e51780 -1 Falling back to public interface 2026-02-12T18:13:48.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-12T18:13:48.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:13:48.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T18:13:48.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:13:48.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:13:48.623 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:13:48.623 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:13:48.623 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:13:48.623 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:13:48.624 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:13:48.835 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:13:48.830+0000 7fa753e51780 -1 osd.1 0 log_to_monitors true 2026-02-12T18:13:48.855 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:13:49.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:13:49.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:13:49.858 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:13:49.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:13:49.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:13:49.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:13:50.084 INFO:tasks.workunit.client.0.trial116.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1651783507,v1:127.0.0.1:6811/1651783507] [v2:127.0.0.1:6812/1651783507,v1:127.0.0.1:6813/1651783507] exists,up 4e37df1a-c5f7-4c64-be85-b14408b1458f 2026-02-12T18:13:50.088 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:13:50.088 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:13:50.088 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:13:50.088 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:124: TEST_ec_profile_blaum_roth_warning: for id in $(seq 0 2) 2026-02-12T18:13:50.088 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:125: TEST_ec_profile_blaum_roth_warning: run_osd td/test-erasure-code-plugins 2 2026-02-12T18:13:50.088 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-12T18:13:50.088 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:13:50.088 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-12T18:13:50.089 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:13:50.089 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/2 2026-02-12T18:13:50.089 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-12T18:13:50.089 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:13:50.089 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:13:50.089 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:13:50.089 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/2' 2026-02-12T18:13:50.089 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/2/journal' 2026-02-12T18:13:50.089 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:13:50.089 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:13:50.089 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-12T18:13:50.089 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:13:50.090 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:13:50.090 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:13:50.090 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:13:50.090 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:13:50.090 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:13:50.090 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:13:50.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:13:50.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:13:50.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:13:50.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:13:50.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-12T18:13:50.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-12T18:13:50.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:13:50.091 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:13:50.091 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:13:50.091 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:13:50.091 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:13:50.091 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:13:50.091 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/2 2026-02-12T18:13:50.091 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:13:50.091 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3fd722a1-0f99-4102-85c0-a04ee451a133 2026-02-12T18:13:50.091 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 3fd722a1-0f99-4102-85c0-a04ee451a133' 2026-02-12T18:13:50.092 INFO:tasks.workunit.client.0.trial116.stdout:add osd2 3fd722a1-0f99-4102-85c0-a04ee451a133 2026-02-12T18:13:50.092 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:13:50.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBeGI5p/U+KBRAAOAb8zLd03BhU5PDAYh+iVA== 2026-02-12T18:13:50.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBeGI5p/U+KBRAAOAb8zLd03BhU5PDAYh+iVA=="}' 2026-02-12T18:13:50.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3fd722a1-0f99-4102-85c0-a04ee451a133 -i td/test-erasure-code-plugins/2/new.json 2026-02-12T18:13:50.326 INFO:tasks.workunit.client.0.trial116.stdout:2 2026-02-12T18:13:50.333 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/2/new.json 2026-02-12T18:13:50.334 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/2 --osd-journal=td/test-erasure-code-plugins/2/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBeGI5p/U+KBRAAOAb8zLd03BhU5PDAYh+iVA== --osd-uuid 3fd722a1-0f99-4102-85c0-a04ee451a133 2026-02-12T18:13:50.348 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:13:50.343+0000 7f525fc35780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:13:50.349 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:13:50.344+0000 7f525fc35780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:13:50.350 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:13:50.345+0000 7f525fc35780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:13:50.350 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:13:50.345+0000 7f525fc35780 -1 bdev(0x3e94c00 td/test-erasure-code-plugins/2/block) open stat got: (1) Operation not permitted 2026-02-12T18:13:50.350 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:13:50.346+0000 7f525fc35780 -1 bluestore(td/test-erasure-code-plugins/2) _read_fsid unparsable uuid 2026-02-12T18:13:50.672 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/2/keyring 2026-02-12T18:13:50.673 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:13:50.673 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-12T18:13:50.674 INFO:tasks.workunit.client.0.trial116.stdout:adding osd2 key to auth repository 2026-02-12T18:13:50.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:13:50.989 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-12T18:13:50.989 INFO:tasks.workunit.client.0.trial116.stdout:start osd.2 2026-02-12T18:13:50.990 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/2 --osd-journal=td/test-erasure-code-plugins/2/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:13:50.990 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:13:50.990 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:13:50.990 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:13:51.001 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:13:50.996+0000 7f218560f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:13:51.003 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:13:50.998+0000 7f218560f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:13:51.003 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:13:50.999+0000 7f218560f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:13:51.147 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:13:51.142+0000 7f218560f780 -1 Falling back to public interface 2026-02-12T18:13:51.214 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-12T18:13:51.214 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:13:51.214 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T18:13:51.214 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:13:51.214 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:13:51.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:13:51.215 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:13:51.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:13:51.216 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:13:51.216 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:13:51.438 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:13:51.465 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:13:51.461+0000 7f218560f780 -1 osd.2 0 log_to_monitors true 2026-02-12T18:13:52.440 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:13:52.440 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:13:52.440 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:13:52.441 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:13:52.441 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:13:52.441 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:13:52.666 INFO:tasks.workunit.client.0.trial116.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3644221357,v1:127.0.0.1:6819/3644221357] [v2:127.0.0.1:6820/3644221357,v1:127.0.0.1:6821/3644221357] exists,up 3fd722a1-0f99-4102-85c0-a04ee451a133 2026-02-12T18:13:52.667 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:13:52.667 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:13:52.667 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:13:52.667 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:127: TEST_ec_profile_blaum_roth_warning: create_rbd_pool 2026-02-12T18:13:52.667 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-12T18:13:52.886 INFO:tasks.workunit.client.0.trial116.stderr:pool 'rbd' does not exist 2026-02-12T18:13:52.893 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-12T18:13:52.894 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T18:13:53.169 INFO:tasks.workunit.client.0.trial116.stderr:pool 'rbd' already exists 2026-02-12T18:13:53.177 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:13:54.178 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-12T18:13:54.458 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:128: TEST_ec_profile_blaum_roth_warning: wait_for_clean 2026-02-12T18:13:54.459 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:13:54.459 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:13:54.459 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:13:54.459 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:13:54.459 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:13:54.459 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:13:54.459 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:13:54.459 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:13:54.459 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:13:54.540 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:13:54.540 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:13:54.540 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:13:54.540 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:13:54.540 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:13:54.540 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:13:54.764 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:13:54.764 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:13:54.764 INFO:tasks.workunit.client.0.trial116.stderr:2' 2026-02-12T18:13:54.764 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:13:54.764 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:13:54.765 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:13:54.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-12T18:13:54.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-12T18:13:54.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484' 2026-02-12T18:13:54.829 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:13:54.829 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:13:54.890 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705667 2026-02-12T18:13:54.890 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705667 2026-02-12T18:13:54.890 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484 1-38654705667' 2026-02-12T18:13:54.891 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:13:54.891 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:13:54.951 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574851 2026-02-12T18:13:54.952 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574851 2026-02-12T18:13:54.952 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484 1-38654705667 2-55834574851' 2026-02-12T18:13:54.952 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:13:54.952 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836484 2026-02-12T18:13:54.953 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:13:54.953 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:13:54.953 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-12T18:13:54.953 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:13:54.954 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836484 2026-02-12T18:13:54.955 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-12T18:13:54.955 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.0 seq 21474836484 2026-02-12T18:13:54.955 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:13:55.181 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-12T18:13:55.181 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:13:56.182 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:13:56.182 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:13:56.407 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-12T18:13:56.407 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:13:56.408 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705667 2026-02-12T18:13:56.408 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:13:56.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:13:56.409 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705667 2026-02-12T18:13:56.410 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:13:56.411 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705667 2026-02-12T18:13:56.411 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.1 seq 38654705667 2026-02-12T18:13:56.411 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705667' 2026-02-12T18:13:56.411 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:13:56.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705668 -lt 38654705667 2026-02-12T18:13:56.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:13:56.639 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574851 2026-02-12T18:13:56.639 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:13:56.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:13:56.641 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574851 2026-02-12T18:13:56.641 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:13:56.642 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574851 2026-02-12T18:13:56.642 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574851' 2026-02-12T18:13:56.642 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.2 seq 55834574851 2026-02-12T18:13:56.642 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:13:56.872 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574851 -lt 55834574851 2026-02-12T18:13:56.872 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:13:56.873 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:13:56.873 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:13:57.181 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-12T18:13:57.181 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:13:57.182 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:13:57.182 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:13:57.182 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:13:57.182 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:13:57.182 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:13:57.182 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:13:57.402 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-12T18:13:57.402 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:13:57.403 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:13:57.403 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:13:57.716 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-12T18:13:57.716 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:13:57.716 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:13:57.716 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:130: TEST_ec_profile_blaum_roth_warning: echo 'Starting test for blaum-roth profile health warning' 2026-02-12T18:13:57.717 INFO:tasks.workunit.client.0.trial116.stdout:Starting test for blaum-roth profile health warning 2026-02-12T18:13:57.717 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:133: TEST_ec_profile_blaum_roth_warning: ceph osd erasure-code-profile set prof- plugin=jerasure k=3 m=1 technique=blaum_roth w=7 --yes-i-really-mean-it --force 2026-02-12T18:13:58.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:135: TEST_ec_profile_blaum_roth_warning: ceph osd erasure-code-profile set test crush-device-class= crush-failure-domain=osd crush-num-failure-domains=0 crush-osds-per-failure-domain=0 crush-root=default k=4 l=3 m=2 name=lrcprofile plugin=lrc 2026-02-12T18:13:58.295 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:136: TEST_ec_profile_blaum_roth_warning: get_asok_path mon.a 2026-02-12T18:13:58.295 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:13:58.295 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:13:58.295 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:13:58.295 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:13:58.296 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:13:58.296 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:13:58.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:136: TEST_ec_profile_blaum_roth_warning: CEPH_ARGS= 2026-02-12T18:13:58.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:136: TEST_ec_profile_blaum_roth_warning: ceph --admin-daemon /tmp/ceph-asok.19237/ceph-mon.a.asok log flush 2026-02-12T18:13:58.335 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:13:58.338 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:137: TEST_ec_profile_blaum_roth_warning: sleep 10 2026-02-12T18:14:08.340 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:138: TEST_ec_profile_blaum_roth_warning: grep -F '1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption' td/test-erasure-code-plugins/mon.a.log 2026-02-12T18:14:08.343 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:00.433+0000 7f20ba3f6640 0 log_channel(cluster) log [WRN] : Health check failed: 1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption (BLAUM_ROTH_W_IS_NOT_PRIME) 2026-02-12T18:14:08.344 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:00.434+0000 7f20b7bf1640 10 mon.a@0(leader).log v76 logging 2026-02-12T18:14:00.433253+0000 mon.a (mon.0) 141 : cluster [WRN] Health check failed: 1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption (BLAUM_ROTH_W_IS_NOT_PRIME) 2026-02-12T18:14:08.344 INFO:tasks.workunit.client.0.trial116.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-12T18:14:08.344 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:00.533+0000 7f20b53ec640 7 mon.a@0(leader).log v77 update_from_paxos applying incremental log 77 2026-02-12T18:14:00.433253+0000 mon.a (mon.0) 141 : cluster [WRN] Health check failed: 1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption (BLAUM_ROTH_W_IS_NOT_PRIME) 2026-02-12T18:14:08.344 INFO:tasks.workunit.client.0.trial116.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-12T18:14:08.344 INFO:tasks.workunit.client.0.trial116.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-12T18:14:08.344 INFO:tasks.workunit.client.0.trial116.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-12T18:14:08.344 INFO:tasks.workunit.client.0.trial116.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-12T18:14:08.344 INFO:tasks.workunit.client.0.trial116.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-12T18:14:08.344 INFO:tasks.workunit.client.0.trial116.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-12T18:14:08.344 INFO:tasks.workunit.client.0.trial116.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-12T18:14:08.345 INFO:tasks.workunit.client.0.trial116.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-12T18:14:08.345 INFO:tasks.workunit.client.0.trial116.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-12T18:14:08.345 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:139: TEST_ec_profile_blaum_roth_warning: grep -F 'w+1=8 for the EC profile prof- is not prime and could lead to data corruption' td/test-erasure-code-plugins/mon.a.log 2026-02-12T18:14:08.347 INFO:tasks.workunit.client.0.trial116.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-12T18:14:08.347 INFO:tasks.workunit.client.0.trial116.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-12T18:14:08.347 INFO:tasks.workunit.client.0.trial116.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-12T18:14:08.347 INFO:tasks.workunit.client.0.trial116.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-12T18:14:08.347 INFO:tasks.workunit.client.0.trial116.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-12T18:14:08.347 INFO:tasks.workunit.client.0.trial116.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-12T18:14:08.347 INFO:tasks.workunit.client.0.trial116.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-12T18:14:08.347 INFO:tasks.workunit.client.0.trial116.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-12T18:14:08.347 INFO:tasks.workunit.client.0.trial116.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-12T18:14:08.348 INFO:tasks.workunit.client.0.trial116.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-12T18:14:08.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:141: TEST_ec_profile_blaum_roth_warning: teardown td/test-erasure-code-plugins 2026-02-12T18:14:08.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:08.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:14:08.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-12T18:14:08.349 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:14:08.349 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:14:08.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:14:08.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:14:08.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:14:08.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:14:08.456 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:14:08.456 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:14:08.457 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:14:08.458 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:14:08.458 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:14:08.459 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:14:08.459 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:08.460 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:14:08.460 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:14:08.460 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:08.461 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:14:08.462 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:14:08.462 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-12T18:14:08.468 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:14:08.468 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:08.468 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:08.468 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19237 2026-02-12T18:14:08.471 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:14:08.471 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:14:08.471 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:142: TEST_ec_profile_blaum_roth_warning: return 0 2026-02-12T18:14:08.471 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-12T18:14:08.471 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_ec_profile_warning td/test-erasure-code-plugins 2026-02-12T18:14:08.471 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:92: TEST_ec_profile_warning: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:08.471 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:94: TEST_ec_profile_warning: setup td/test-erasure-code-plugins 2026-02-12T18:14:08.471 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:08.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-12T18:14:08.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:08.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:14:08.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-12T18:14:08.472 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:14:08.472 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:14:08.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:14:08.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:14:08.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:14:08.473 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:14:08.473 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:14:08.473 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:14:08.473 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:14:08.474 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:14:08.474 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:14:08.474 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:14:08.474 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:08.475 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:14:08.475 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:14:08.475 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:08.475 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:14:08.476 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:14:08.476 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-12T18:14:08.477 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:14:08.477 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:08.478 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:08.478 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19237 2026-02-12T18:14:08.478 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:14:08.478 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:14:08.478 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-12T18:14:08.479 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T18:14:08.480 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:08.480 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:08.480 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19237 2026-02-12T18:14:08.481 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T18:14:08.482 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T18:14:08.482 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T18:14:08.482 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-12T18:14:08.482 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:95: TEST_ec_profile_warning: run_mon td/test-erasure-code-plugins a 2026-02-12T18:14:08.482 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:08.482 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T18:14:08.482 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T18:14:08.483 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T18:14:08.483 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-12T18:14:08.483 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins 2026-02-12T18:14:08.499 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T18:14:08.499 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:08.499 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:08.499 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:08.499 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:08.500 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:08.500 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:08.500 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T18:14:08.522 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T18:14:08.523 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T18:14:08.523 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:14:08.523 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:14:08.523 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T18:14:08.523 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T18:14:08.525 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:14:08.525 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:08.525 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:08.525 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:08.525 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:08.525 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:08.525 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:08.525 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:14:08.525 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19237/ceph-mon.a.asok config get fsid 2026-02-12T18:14:08.567 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T18:14:08.567 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:14:08.567 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:14:08.568 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T18:14:08.568 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T18:14:08.568 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:14:08.568 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:08.568 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:08.569 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:08.569 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:08.569 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:08.569 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:08.569 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:14:08.569 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19237/ceph-mon.a.asok config get mon_host 2026-02-12T18:14:08.612 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:96: TEST_ec_profile_warning: run_mgr td/test-erasure-code-plugins x 2026-02-12T18:14:08.612 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:08.612 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T18:14:08.612 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T18:14:08.612 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T18:14:08.612 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-12T18:14:08.612 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T18:14:08.710 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T18:14:08.711 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:08.711 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:08.711 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:08.711 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:08.711 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:08.711 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:08.712 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:14:08.712 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:14:08.725 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: seq 0 2 2026-02-12T18:14:08.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: for id in $(seq 0 2) 2026-02-12T18:14:08.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:98: TEST_ec_profile_warning: run_osd td/test-erasure-code-plugins 0 2026-02-12T18:14:08.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:08.730 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:14:08.730 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T18:14:08.730 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:14:08.730 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-12T18:14:08.730 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-12T18:14:08.730 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:14:08.730 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:14:08.730 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:14:08.730 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-12T18:14:08.730 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-12T18:14:08.730 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:14:08.730 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:14:08.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-12T18:14:08.731 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:14:08.731 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:08.731 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:08.731 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:08.731 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:08.731 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:08.731 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:08.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:08.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:14:08.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:14:08.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:14:08.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-12T18:14:08.732 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-12T18:14:08.732 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:14:08.732 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:14:08.732 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:14:08.732 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:14:08.732 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:14:08.732 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:14:08.732 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-12T18:14:08.732 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:14:08.732 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=10e0b890-c2b0-4f4a-bb7e-0821fbe6ed8f 2026-02-12T18:14:08.732 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 10e0b890-c2b0-4f4a-bb7e-0821fbe6ed8f' 2026-02-12T18:14:08.733 INFO:tasks.workunit.client.0.trial116.stdout:add osd0 10e0b890-c2b0-4f4a-bb7e-0821fbe6ed8f 2026-02-12T18:14:08.733 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:14:08.738 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBwGI5p+uK/KxAAztohWlkFscjUZMDwHZu63Q== 2026-02-12T18:14:08.738 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBwGI5p+uK/KxAAztohWlkFscjUZMDwHZu63Q=="}' 2026-02-12T18:14:08.738 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 10e0b890-c2b0-4f4a-bb7e-0821fbe6ed8f -i td/test-erasure-code-plugins/0/new.json 2026-02-12T18:14:08.838 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:08.844 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-12T18:14:08.845 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBwGI5p+uK/KxAAztohWlkFscjUZMDwHZu63Q== --osd-uuid 10e0b890-c2b0-4f4a-bb7e-0821fbe6ed8f 2026-02-12T18:14:08.860 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:08.857+0000 7f1a5b20f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:08.862 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:08.858+0000 7f1a5b20f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:08.862 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:08.859+0000 7f1a5b20f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:08.863 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:08.859+0000 7f1a5b20f780 -1 bdev(0x2934c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-12T18:14:08.863 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:08.859+0000 7f1a5b20f780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-12T18:14:09.269 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-12T18:14:09.269 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:14:09.270 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T18:14:09.270 INFO:tasks.workunit.client.0.trial116.stdout:adding osd0 key to auth repository 2026-02-12T18:14:09.271 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:14:09.370 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T18:14:09.370 INFO:tasks.workunit.client.0.trial116.stdout:start osd.0 2026-02-12T18:14:09.371 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:14:09.371 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:14:09.371 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:14:09.371 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:14:09.383 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:09.380+0000 7f3fbc443780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:09.385 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:09.381+0000 7f3fbc443780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:09.386 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:09.382+0000 7f3fbc443780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:09.466 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T18:14:09.467 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:14:09.467 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T18:14:09.467 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:14:09.467 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:14:09.467 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:09.467 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:14:09.467 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:09.468 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:09.468 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:14:09.561 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:14:09.596 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:09.593+0000 7f3fbc443780 -1 Falling back to public interface 2026-02-12T18:14:10.011 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:10.007+0000 7f3fbc443780 -1 osd.0 0 log_to_monitors true 2026-02-12T18:14:10.563 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:14:10.563 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:10.563 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:14:10.564 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:14:10.564 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:10.565 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:14:10.657 INFO:tasks.workunit.client.0.trial116.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2784689337,v1:127.0.0.1:6801/2784689337] [v2:127.0.0.1:6802/2784689337,v1:127.0.0.1:6803/2784689337] exists,up 10e0b890-c2b0-4f4a-bb7e-0821fbe6ed8f 2026-02-12T18:14:10.661 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:14:10.661 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:14:10.661 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:14:10.661 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: for id in $(seq 0 2) 2026-02-12T18:14:10.661 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:98: TEST_ec_profile_warning: run_osd td/test-erasure-code-plugins 1 2026-02-12T18:14:10.661 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:10.662 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:14:10.662 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-12T18:14:10.662 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:14:10.662 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/1 2026-02-12T18:14:10.662 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-12T18:14:10.662 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:14:10.662 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:14:10.662 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:14:10.662 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/1' 2026-02-12T18:14:10.662 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/1/journal' 2026-02-12T18:14:10.662 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:14:10.663 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:14:10.663 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-12T18:14:10.663 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:14:10.663 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:10.663 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:10.663 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:10.663 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:10.663 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:10.663 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:10.663 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:10.663 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:14:10.663 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:14:10.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:14:10.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-12T18:14:10.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-12T18:14:10.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:14:10.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:14:10.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:14:10.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:14:10.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:14:10.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:14:10.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/1 2026-02-12T18:14:10.664 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:14:10.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5f3e0839-ea2c-4600-b365-ac4bf90c94e2 2026-02-12T18:14:10.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 5f3e0839-ea2c-4600-b365-ac4bf90c94e2' 2026-02-12T18:14:10.665 INFO:tasks.workunit.client.0.trial116.stdout:add osd1 5f3e0839-ea2c-4600-b365-ac4bf90c94e2 2026-02-12T18:14:10.665 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:14:10.670 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQByGI5p4vCvJxAABF9z4SKWlf6KnLtmhPNJdg== 2026-02-12T18:14:10.670 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQByGI5p4vCvJxAABF9z4SKWlf6KnLtmhPNJdg=="}' 2026-02-12T18:14:10.670 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5f3e0839-ea2c-4600-b365-ac4bf90c94e2 -i td/test-erasure-code-plugins/1/new.json 2026-02-12T18:14:10.771 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:14:10.776 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/1/new.json 2026-02-12T18:14:10.777 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/1 --osd-journal=td/test-erasure-code-plugins/1/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQByGI5p4vCvJxAABF9z4SKWlf6KnLtmhPNJdg== --osd-uuid 5f3e0839-ea2c-4600-b365-ac4bf90c94e2 2026-02-12T18:14:10.792 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:10.788+0000 7f5da640e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:10.796 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:10.792+0000 7f5da640e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:10.797 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:10.793+0000 7f5da640e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:10.797 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:10.794+0000 7f5da640e780 -1 bdev(0x3330c00 td/test-erasure-code-plugins/1/block) open stat got: (1) Operation not permitted 2026-02-12T18:14:10.797 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:10.794+0000 7f5da640e780 -1 bluestore(td/test-erasure-code-plugins/1) _read_fsid unparsable uuid 2026-02-12T18:14:11.127 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/1/keyring 2026-02-12T18:14:11.128 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:14:11.128 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-12T18:14:11.129 INFO:tasks.workunit.client.0.trial116.stdout:adding osd1 key to auth repository 2026-02-12T18:14:11.129 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:14:11.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-12T18:14:11.455 INFO:tasks.workunit.client.0.trial116.stdout:start osd.1 2026-02-12T18:14:11.456 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/1 --osd-journal=td/test-erasure-code-plugins/1/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:14:11.456 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:14:11.456 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:14:11.456 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:14:11.468 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:11.465+0000 7f68b5429780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:11.469 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:11.466+0000 7f68b5429780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:11.470 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:11.467+0000 7f68b5429780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:11.593 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:11.589+0000 7f68b5429780 -1 Falling back to public interface 2026-02-12T18:14:11.679 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-12T18:14:11.679 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:14:11.679 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T18:14:11.679 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:14:11.679 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:14:11.679 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:11.680 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:11.680 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:14:11.680 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:11.680 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:14:11.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:14:11.907 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:11.902+0000 7f68b5429780 -1 osd.1 0 log_to_monitors true 2026-02-12T18:14:12.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:14:12.905 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:12.905 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:14:12.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:14:12.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:12.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:14:13.131 INFO:tasks.workunit.client.0.trial116.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3680627562,v1:127.0.0.1:6811/3680627562] [v2:127.0.0.1:6812/3680627562,v1:127.0.0.1:6813/3680627562] exists,up 5f3e0839-ea2c-4600-b365-ac4bf90c94e2 2026-02-12T18:14:13.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:14:13.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:14:13.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:14:13.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: for id in $(seq 0 2) 2026-02-12T18:14:13.136 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:98: TEST_ec_profile_warning: run_osd td/test-erasure-code-plugins 2 2026-02-12T18:14:13.136 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:13.136 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:14:13.136 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-12T18:14:13.136 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:14:13.136 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/2 2026-02-12T18:14:13.136 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-12T18:14:13.136 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:14:13.136 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:14:13.136 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:14:13.136 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/2' 2026-02-12T18:14:13.136 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/2/journal' 2026-02-12T18:14:13.137 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:14:13.137 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:14:13.137 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-12T18:14:13.137 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:14:13.137 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:13.137 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:13.137 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:13.137 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:13.137 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:13.137 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:13.137 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:13.137 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:14:13.138 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:14:13.138 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:14:13.138 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-12T18:14:13.138 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-12T18:14:13.138 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:14:13.138 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:14:13.138 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:14:13.138 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:14:13.138 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:14:13.138 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:14:13.138 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/2 2026-02-12T18:14:13.138 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:14:13.138 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f1a14430-3843-437a-94a9-1de41446cce0 2026-02-12T18:14:13.139 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 f1a14430-3843-437a-94a9-1de41446cce0' 2026-02-12T18:14:13.139 INFO:tasks.workunit.client.0.trial116.stdout:add osd2 f1a14430-3843-437a-94a9-1de41446cce0 2026-02-12T18:14:13.139 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:14:13.144 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB1GI5pn2ZXCBAAtPmNAw3K2MbqN5QiRkrJgg== 2026-02-12T18:14:13.144 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB1GI5pn2ZXCBAAtPmNAw3K2MbqN5QiRkrJgg=="}' 2026-02-12T18:14:13.144 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f1a14430-3843-437a-94a9-1de41446cce0 -i td/test-erasure-code-plugins/2/new.json 2026-02-12T18:14:13.369 INFO:tasks.workunit.client.0.trial116.stdout:2 2026-02-12T18:14:13.377 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/2/new.json 2026-02-12T18:14:13.378 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/2 --osd-journal=td/test-erasure-code-plugins/2/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB1GI5pn2ZXCBAAtPmNAw3K2MbqN5QiRkrJgg== --osd-uuid f1a14430-3843-437a-94a9-1de41446cce0 2026-02-12T18:14:13.392 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:13.387+0000 7f6e9ec27780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:13.393 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:13.389+0000 7f6e9ec27780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:13.394 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:13.390+0000 7f6e9ec27780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:13.395 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:13.390+0000 7f6e9ec27780 -1 bdev(0x2848c00 td/test-erasure-code-plugins/2/block) open stat got: (1) Operation not permitted 2026-02-12T18:14:13.395 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:13.390+0000 7f6e9ec27780 -1 bluestore(td/test-erasure-code-plugins/2) _read_fsid unparsable uuid 2026-02-12T18:14:13.721 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/2/keyring 2026-02-12T18:14:13.721 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:14:13.721 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-12T18:14:13.722 INFO:tasks.workunit.client.0.trial116.stdout:adding osd2 key to auth repository 2026-02-12T18:14:13.722 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:14:14.044 INFO:tasks.workunit.client.0.trial116.stdout:start osd.2 2026-02-12T18:14:14.045 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-12T18:14:14.045 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/2 --osd-journal=td/test-erasure-code-plugins/2/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:14:14.045 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:14:14.045 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:14:14.045 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:14:14.057 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:14.053+0000 7f695c82d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:14.059 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:14.054+0000 7f695c82d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:14.059 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:14.055+0000 7f695c82d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:14.232 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:14.227+0000 7f695c82d780 -1 Falling back to public interface 2026-02-12T18:14:14.270 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-12T18:14:14.270 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:14:14.270 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T18:14:14.271 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:14:14.271 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:14:14.271 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:14.271 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:14:14.271 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:14.272 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:14.272 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:14:14.492 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:14:14.582 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:14.578+0000 7f695c82d780 -1 osd.2 0 log_to_monitors true 2026-02-12T18:14:15.493 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:14:15.494 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:15.494 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:14:15.495 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:14:15.495 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:15.495 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:14:15.721 INFO:tasks.workunit.client.0.trial116.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2458075307,v1:127.0.0.1:6819/2458075307] [v2:127.0.0.1:6820/2458075307,v1:127.0.0.1:6821/2458075307] exists,up f1a14430-3843-437a-94a9-1de41446cce0 2026-02-12T18:14:15.722 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:14:15.722 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:14:15.722 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:14:15.722 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:100: TEST_ec_profile_warning: create_rbd_pool 2026-02-12T18:14:15.722 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-12T18:14:15.941 INFO:tasks.workunit.client.0.trial116.stderr:pool 'rbd' does not exist 2026-02-12T18:14:15.949 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-12T18:14:15.949 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T18:14:16.237 INFO:tasks.workunit.client.0.trial116.stderr:pool 'rbd' already exists 2026-02-12T18:14:16.245 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:14:17.246 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-12T18:14:17.527 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:101: TEST_ec_profile_warning: wait_for_clean 2026-02-12T18:14:17.527 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:14:17.527 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:14:17.528 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:14:17.528 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:14:17.528 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:14:17.528 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:14:17.528 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:14:17.528 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:14:17.528 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:14:17.608 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:14:17.608 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:14:17.608 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:14:17.608 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:14:17.608 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:14:17.608 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:14:17.842 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:14:17.843 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:14:17.843 INFO:tasks.workunit.client.0.trial116.stderr:2' 2026-02-12T18:14:17.843 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:14:17.843 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:14:17.843 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:14:17.907 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-12T18:14:17.907 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-12T18:14:17.907 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484' 2026-02-12T18:14:17.907 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:14:17.907 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:14:17.969 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705668 2026-02-12T18:14:17.969 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705668 2026-02-12T18:14:17.970 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484 1-38654705668' 2026-02-12T18:14:17.970 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:14:17.970 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:14:18.032 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574851 2026-02-12T18:14:18.032 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574851 2026-02-12T18:14:18.033 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484 1-38654705668 2-55834574851' 2026-02-12T18:14:18.033 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:14:18.033 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836484 2026-02-12T18:14:18.033 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:14:18.034 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:14:18.034 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-12T18:14:18.034 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:14:18.035 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836484 2026-02-12T18:14:18.036 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-12T18:14:18.036 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.0 seq 21474836484 2026-02-12T18:14:18.036 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:14:18.258 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-12T18:14:18.258 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:14:19.259 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:14:19.260 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:14:19.484 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-12T18:14:19.484 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:14:19.485 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705668 2026-02-12T18:14:19.485 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:14:19.486 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:14:19.486 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705668 2026-02-12T18:14:19.486 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:14:19.487 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705668 2026-02-12T18:14:19.488 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.1 seq 38654705668 2026-02-12T18:14:19.488 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705668' 2026-02-12T18:14:19.488 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:14:19.714 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705669 -lt 38654705668 2026-02-12T18:14:19.714 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:14:19.715 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574851 2026-02-12T18:14:19.715 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:14:19.716 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:14:19.716 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574851 2026-02-12T18:14:19.716 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:14:19.717 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574851 2026-02-12T18:14:19.718 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574851' 2026-02-12T18:14:19.718 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.2 seq 55834574851 2026-02-12T18:14:19.718 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:14:19.943 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574851 -lt 55834574851 2026-02-12T18:14:19.944 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:14:19.944 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:14:19.944 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:14:20.257 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-12T18:14:20.258 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:14:20.258 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:14:20.258 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:14:20.258 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:14:20.258 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:14:20.258 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:14:20.258 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:14:20.479 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-12T18:14:20.480 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:14:20.480 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:14:20.480 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:14:20.795 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-12T18:14:20.795 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:14:20.796 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:14:20.796 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:103: TEST_ec_profile_warning: for plugin in ${legacy_jerasure_plugins[*]} 2026-02-12T18:14:20.796 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:104: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-jerasure_generic crush-failure-domain=osd technique=reed_sol_van plugin=jerasure_generic 2026-02-12T18:14:21.082 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-12T18:14:21.082 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:21.082 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:21.083 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:21.083 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:21.083 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:21.083 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:21.083 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-12T18:14:21.083 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.19237/ceph-mon.a.asok log flush 2026-02-12T18:14:21.121 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:14:21.124 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:106: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-12T18:14:21.127 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:20.951+0000 7f61437f1640 0 mon.a@0(leader).osd e19 WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic that has been deprecated. Please use jerasure instead. 2026-02-12T18:14:21.127 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:21.068+0000 7f61437f1640 0 mon.a@0(leader).osd e20 WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic that has been deprecated. Please use jerasure instead. 2026-02-12T18:14:21.127 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:21.068+0000 7f61437f1640 0 mon.a@0(leader).osd e20 WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic that has been deprecated. Please use jerasure instead. 2026-02-12T18:14:21.127 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:103: TEST_ec_profile_warning: for plugin in ${legacy_jerasure_plugins[*]} 2026-02-12T18:14:21.127 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:104: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-jerasure_sse3 crush-failure-domain=osd technique=reed_sol_van plugin=jerasure_sse3 2026-02-12T18:14:21.415 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-12T18:14:21.415 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:21.416 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:21.416 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:21.416 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:21.416 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:21.416 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:21.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-12T18:14:21.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.19237/ceph-mon.a.asok log flush 2026-02-12T18:14:21.454 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:14:21.458 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:106: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-12T18:14:21.460 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:21.283+0000 7f61437f1640 0 mon.a@0(leader).osd e20 WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3 that has been deprecated. Please use jerasure instead. 2026-02-12T18:14:21.460 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:21.402+0000 7f61437f1640 0 mon.a@0(leader).osd e21 WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3 that has been deprecated. Please use jerasure instead. 2026-02-12T18:14:21.460 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:21.402+0000 7f61437f1640 0 mon.a@0(leader).osd e21 WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3 that has been deprecated. Please use jerasure instead. 2026-02-12T18:14:21.461 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:103: TEST_ec_profile_warning: for plugin in ${legacy_jerasure_plugins[*]} 2026-02-12T18:14:21.461 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:104: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-jerasure_sse4 crush-failure-domain=osd technique=reed_sol_van plugin=jerasure_sse4 2026-02-12T18:14:21.749 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-12T18:14:21.749 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:21.749 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:21.749 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:21.749 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:21.749 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:21.750 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:21.750 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-12T18:14:21.750 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.19237/ceph-mon.a.asok log flush 2026-02-12T18:14:21.790 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:14:21.795 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:106: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-12T18:14:21.797 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:21.617+0000 7f61437f1640 0 mon.a@0(leader).osd e21 WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4 that has been deprecated. Please use jerasure instead. 2026-02-12T18:14:21.797 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:21.736+0000 7f61437f1640 0 mon.a@0(leader).osd e22 WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4 that has been deprecated. Please use jerasure instead. 2026-02-12T18:14:21.798 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:21.736+0000 7f61437f1640 0 mon.a@0(leader).osd e22 WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4 that has been deprecated. Please use jerasure instead. 2026-02-12T18:14:21.798 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:109: TEST_ec_profile_warning: for plugin in ${legacy_shec_plugins[*]} 2026-02-12T18:14:21.798 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:110: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-shec_generic crush-failure-domain=osd plugin=shec_generic 2026-02-12T18:14:22.088 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-12T18:14:22.088 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:22.088 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:22.088 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:22.089 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:22.089 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:22.089 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:22.089 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-12T18:14:22.089 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.19237/ceph-mon.a.asok log flush 2026-02-12T18:14:22.126 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:14:22.130 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:112: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-12T18:14:22.133 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:21.955+0000 7f61437f1640 0 mon.a@0(leader).osd e22 WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic that has been deprecated. Please use shec instead. 2026-02-12T18:14:22.133 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:22.075+0000 7f61437f1640 0 mon.a@0(leader).osd e23 WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic that has been deprecated. Please use shec instead. 2026-02-12T18:14:22.133 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:22.075+0000 7f61437f1640 0 mon.a@0(leader).osd e23 WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic that has been deprecated. Please use shec instead. 2026-02-12T18:14:22.133 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:109: TEST_ec_profile_warning: for plugin in ${legacy_shec_plugins[*]} 2026-02-12T18:14:22.133 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:110: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-shec_sse3 crush-failure-domain=osd plugin=shec_sse3 2026-02-12T18:14:22.426 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-12T18:14:22.426 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:22.426 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:22.426 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:22.426 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:22.427 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:22.427 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:22.427 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-12T18:14:22.427 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.19237/ceph-mon.a.asok log flush 2026-02-12T18:14:22.464 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:14:22.468 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:112: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-12T18:14:22.470 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:22.294+0000 7f61437f1640 0 mon.a@0(leader).osd e23 WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3 that has been deprecated. Please use shec instead. 2026-02-12T18:14:22.470 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:22.412+0000 7f61437f1640 0 mon.a@0(leader).osd e24 WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3 that has been deprecated. Please use shec instead. 2026-02-12T18:14:22.471 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:22.412+0000 7f61437f1640 0 mon.a@0(leader).osd e24 WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3 that has been deprecated. Please use shec instead. 2026-02-12T18:14:22.471 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:109: TEST_ec_profile_warning: for plugin in ${legacy_shec_plugins[*]} 2026-02-12T18:14:22.471 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:110: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-shec_sse4 crush-failure-domain=osd plugin=shec_sse4 2026-02-12T18:14:22.759 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-12T18:14:22.759 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:22.759 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:22.759 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:22.760 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:22.760 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:22.760 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:22.760 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-12T18:14:22.760 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.19237/ceph-mon.a.asok log flush 2026-02-12T18:14:22.798 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:14:22.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:112: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-12T18:14:22.803 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:22.627+0000 7f61437f1640 0 mon.a@0(leader).osd e24 WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4 that has been deprecated. Please use shec instead. 2026-02-12T18:14:22.804 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:22.745+0000 7f61437f1640 0 mon.a@0(leader).osd e25 WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4 that has been deprecated. Please use shec instead. 2026-02-12T18:14:22.804 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:22.745+0000 7f61437f1640 0 mon.a@0(leader).osd e25 WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4 that has been deprecated. Please use shec instead. 2026-02-12T18:14:22.805 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:115: TEST_ec_profile_warning: teardown td/test-erasure-code-plugins 2026-02-12T18:14:22.805 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:22.805 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:14:22.805 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-12T18:14:22.805 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:14:22.805 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:14:22.805 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:14:22.805 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:14:22.805 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:14:22.912 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:14:22.912 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:14:22.913 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:14:22.913 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:14:22.915 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:14:22.915 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:14:22.915 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:14:22.916 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:22.916 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:14:22.917 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:14:22.917 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:22.917 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:14:22.918 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:14:22.918 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-12T18:14:22.925 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:14:22.925 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:22.925 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:22.925 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19237 2026-02-12T18:14:22.927 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:14:22.927 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:14:22.927 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-12T18:14:22.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_preload_no_warning td/test-erasure-code-plugins 2026-02-12T18:14:22.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:58: TEST_preload_no_warning: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:22.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-12T18:14:22.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-12T18:14:22.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:22.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-12T18:14:22.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:22.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:14:22.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-12T18:14:22.928 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:14:22.929 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:14:22.929 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:14:22.929 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:14:22.929 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:14:22.929 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:14:22.930 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:14:22.930 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:14:22.930 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:14:22.931 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:14:22.931 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:14:22.931 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:14:22.932 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:22.932 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:14:22.932 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:14:22.932 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:22.933 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:14:22.934 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:14:22.934 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-12T18:14:22.935 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:14:22.935 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:22.935 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:22.936 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19237 2026-02-12T18:14:22.936 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:14:22.936 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:14:22.936 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-12T18:14:22.937 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T18:14:22.937 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:22.938 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:22.938 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19237 2026-02-12T18:14:22.938 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T18:14:22.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T18:14:22.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T18:14:22.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-12T18:14:22.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure 2026-02-12T18:14:22.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:22.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T18:14:22.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T18:14:22.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T18:14:22.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-12T18:14:22.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure 2026-02-12T18:14:22.957 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T18:14:22.957 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:22.957 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:22.957 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:22.957 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:22.957 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:22.957 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:22.957 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure 2026-02-12T18:14:22.981 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T18:14:22.982 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T18:14:22.982 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:14:22.983 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:14:22.983 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T18:14:22.983 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:14:22.983 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T18:14:22.983 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:22.983 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:22.983 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:22.983 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:22.983 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:22.984 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:22.984 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:14:22.984 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19237/ceph-mon.a.asok config get fsid 2026-02-12T18:14:23.028 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T18:14:23.028 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:14:23.028 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:14:23.028 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T18:14:23.028 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T18:14:23.028 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:14:23.029 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:23.029 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:23.030 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:23.030 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:23.030 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:23.030 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:23.030 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:14:23.030 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19237/ceph-mon.a.asok config get mon_host 2026-02-12T18:14:23.069 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-12T18:14:23.070 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:23.070 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T18:14:23.070 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T18:14:23.070 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T18:14:23.070 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-12T18:14:23.070 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T18:14:23.169 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T18:14:23.169 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:23.169 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:23.169 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:23.169 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:23.170 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:23.170 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:23.170 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:14:23.170 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:14:23.186 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-12T18:14:23.186 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:23.186 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:23.186 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:23.186 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:23.186 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:23.186 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:23.186 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-12T18:14:23.187 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.19237/ceph-mon.a.asok log flush 2026-02-12T18:14:23.224 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:14:23.231 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure 2026-02-12T18:14:23.231 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:23.231 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:14:23.231 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T18:14:23.231 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:14:23.231 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-12T18:14:23.232 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-12T18:14:23.232 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:14:23.232 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:14:23.232 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:14:23.232 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-12T18:14:23.232 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-12T18:14:23.232 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:14:23.232 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:14:23.232 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-12T18:14:23.232 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:14:23.232 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:23.232 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:23.232 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:23.233 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:23.233 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:23.233 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:23.233 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:23.233 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:14:23.233 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:14:23.233 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:14:23.233 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-12T18:14:23.233 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-12T18:14:23.233 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:14:23.233 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:14:23.233 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:14:23.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:14:23.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:14:23.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure 2026-02-12T18:14:23.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-12T18:14:23.234 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:14:23.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c857aae5-14a0-49ad-898e-b8aad994e79f 2026-02-12T18:14:23.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 c857aae5-14a0-49ad-898e-b8aad994e79f' 2026-02-12T18:14:23.234 INFO:tasks.workunit.client.0.trial116.stdout:add osd0 c857aae5-14a0-49ad-898e-b8aad994e79f 2026-02-12T18:14:23.235 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:14:23.240 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB/GI5pQ3sbDhAA3Pb2hpRC4ttyE08t/CsB9A== 2026-02-12T18:14:23.241 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB/GI5pQ3sbDhAA3Pb2hpRC4ttyE08t/CsB9A=="}' 2026-02-12T18:14:23.241 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c857aae5-14a0-49ad-898e-b8aad994e79f -i td/test-erasure-code-plugins/0/new.json 2026-02-12T18:14:23.343 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:23.350 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-12T18:14:23.351 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure --mkfs --key AQB/GI5pQ3sbDhAA3Pb2hpRC4ttyE08t/CsB9A== --osd-uuid c857aae5-14a0-49ad-898e-b8aad994e79f 2026-02-12T18:14:23.366 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:23.362+0000 7f8b90011780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:23.367 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:23.363+0000 7f8b90011780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:23.368 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:23.364+0000 7f8b90011780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:23.369 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:23.364+0000 7f8b90011780 -1 bdev(0x3708c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-12T18:14:23.369 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:23.364+0000 7f8b90011780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-12T18:14:23.804 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-12T18:14:23.804 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:14:23.805 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T18:14:23.805 INFO:tasks.workunit.client.0.trial116.stdout:adding osd0 key to auth repository 2026-02-12T18:14:23.806 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:14:23.907 INFO:tasks.workunit.client.0.trial116.stdout:start osd.0 2026-02-12T18:14:23.908 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T18:14:23.908 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure 2026-02-12T18:14:23.908 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:14:23.908 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:14:23.908 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:14:23.920 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:23.916+0000 7f9399238780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:23.922 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:23.918+0000 7f9399238780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:23.923 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:23.918+0000 7f9399238780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:24.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T18:14:24.008 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:14:24.008 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T18:14:24.008 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:14:24.008 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:14:24.008 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:24.008 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:24.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:14:24.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:24.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:14:24.107 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:14:24.120 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:24.115+0000 7f9399238780 -1 Falling back to public interface 2026-02-12T18:14:24.442 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:24.437+0000 7f9399238780 -1 osd.0 0 log_to_monitors true 2026-02-12T18:14:25.108 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:14:25.108 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:25.109 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:14:25.109 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:14:25.110 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:25.110 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:14:25.205 INFO:tasks.workunit.client.0.trial116.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/25916540,v1:127.0.0.1:6801/25916540] [v2:127.0.0.1:6802/25916540,v1:127.0.0.1:6803/25916540] exists,up c857aae5-14a0-49ad-898e-b8aad994e79f 2026-02-12T18:14:25.206 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:14:25.207 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:14:25.207 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:14:25.207 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-12T18:14:25.207 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T18:14:25.207 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T18:14:25.207 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:25.207 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:25.207 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:25.207 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-osd.0.asok 2026-02-12T18:14:25.208 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-12T18:14:25.208 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.19237/ceph-osd.0.asok log flush 2026-02-12T18:14:25.252 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:14:25.256 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-12T18:14:25.258 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-12T18:14:25.260 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-12T18:14:25.260 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:25.260 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:14:25.260 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-12T18:14:25.260 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:14:25.260 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:14:25.260 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:14:25.260 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:14:25.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:14:25.367 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:14:25.367 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:14:25.368 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:14:25.369 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:14:25.370 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:14:25.370 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:14:25.371 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:14:25.371 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:25.372 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:14:25.372 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:14:25.372 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:25.373 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:14:25.374 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:14:25.374 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-12T18:14:25.377 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:14:25.377 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:25.377 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:25.377 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19237 2026-02-12T18:14:25.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:14:25.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:14:25.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-12T18:14:25.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-12T18:14:25.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:25.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-12T18:14:25.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:25.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:14:25.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-12T18:14:25.379 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:14:25.380 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:14:25.380 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:14:25.380 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:14:25.380 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:14:25.380 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:14:25.380 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:14:25.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:14:25.381 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:14:25.382 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:14:25.382 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:14:25.382 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:14:25.383 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:25.383 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:14:25.383 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:14:25.383 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:25.384 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:14:25.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:14:25.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-12T18:14:25.386 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:14:25.386 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:25.386 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:25.386 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19237 2026-02-12T18:14:25.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:14:25.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:14:25.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-12T18:14:25.388 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T18:14:25.388 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:25.388 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:25.388 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19237 2026-02-12T18:14:25.389 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T18:14:25.390 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T18:14:25.391 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T18:14:25.391 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-12T18:14:25.391 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec 2026-02-12T18:14:25.391 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:25.391 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T18:14:25.391 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T18:14:25.391 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T18:14:25.391 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-12T18:14:25.391 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec 2026-02-12T18:14:25.409 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T18:14:25.409 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:25.409 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:25.409 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:25.409 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:25.409 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:25.409 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:25.410 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec 2026-02-12T18:14:25.432 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T18:14:25.433 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T18:14:25.433 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:14:25.433 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:14:25.433 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T18:14:25.433 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T18:14:25.434 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:14:25.434 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:25.434 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:25.434 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:25.434 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:25.434 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:25.434 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:25.435 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:14:25.435 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19237/ceph-mon.a.asok config get fsid 2026-02-12T18:14:25.478 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T18:14:25.478 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:14:25.478 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:14:25.479 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T18:14:25.479 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T18:14:25.479 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:14:25.479 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:25.479 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:25.479 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:25.479 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:25.479 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:25.480 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:25.480 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:14:25.480 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19237/ceph-mon.a.asok config get mon_host 2026-02-12T18:14:25.523 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-12T18:14:25.523 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:25.523 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T18:14:25.523 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T18:14:25.524 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T18:14:25.524 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-12T18:14:25.524 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T18:14:25.624 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T18:14:25.624 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:25.624 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:25.624 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:25.624 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:25.624 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:25.624 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:25.624 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:14:25.625 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:14:25.639 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-12T18:14:25.639 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:25.639 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:25.639 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:25.639 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:25.639 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:25.639 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:25.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-12T18:14:25.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.19237/ceph-mon.a.asok log flush 2026-02-12T18:14:25.677 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:14:25.684 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec 2026-02-12T18:14:25.685 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:25.685 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:14:25.685 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T18:14:25.685 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:14:25.685 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-12T18:14:25.685 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-12T18:14:25.685 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:14:25.685 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:14:25.685 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:14:25.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-12T18:14:25.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-12T18:14:25.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:14:25.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:14:25.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-12T18:14:25.686 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:14:25.686 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:25.686 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:25.686 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:25.686 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:25.686 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:25.686 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:25.687 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:25.687 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:14:25.687 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:14:25.687 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:14:25.687 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-12T18:14:25.687 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-12T18:14:25.687 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:14:25.687 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:14:25.687 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:14:25.687 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:14:25.687 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:14:25.687 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec 2026-02-12T18:14:25.688 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-12T18:14:25.688 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:14:25.688 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6afddd7c-2049-4fc5-9aea-1d2b093a9399 2026-02-12T18:14:25.688 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 6afddd7c-2049-4fc5-9aea-1d2b093a9399' 2026-02-12T18:14:25.688 INFO:tasks.workunit.client.0.trial116.stdout:add osd0 6afddd7c-2049-4fc5-9aea-1d2b093a9399 2026-02-12T18:14:25.689 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:14:25.694 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCBGI5ps8omKRAAIyJHsTAUR452/mPueXaSLA== 2026-02-12T18:14:25.694 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCBGI5ps8omKRAAIyJHsTAUR452/mPueXaSLA=="}' 2026-02-12T18:14:25.694 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6afddd7c-2049-4fc5-9aea-1d2b093a9399 -i td/test-erasure-code-plugins/0/new.json 2026-02-12T18:14:25.795 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:25.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-12T18:14:25.803 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec --mkfs --key AQCBGI5ps8omKRAAIyJHsTAUR452/mPueXaSLA== --osd-uuid 6afddd7c-2049-4fc5-9aea-1d2b093a9399 2026-02-12T18:14:25.819 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:25.814+0000 7fd167e34780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:25.820 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:25.816+0000 7fd167e34780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:25.821 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:25.817+0000 7fd167e34780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:25.822 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:25.817+0000 7fd167e34780 -1 bdev(0x33d8c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-12T18:14:25.822 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:25.817+0000 7fd167e34780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-12T18:14:26.233 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-12T18:14:26.233 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:14:26.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T18:14:26.235 INFO:tasks.workunit.client.0.trial116.stdout:adding osd0 key to auth repository 2026-02-12T18:14:26.235 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:14:26.335 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T18:14:26.335 INFO:tasks.workunit.client.0.trial116.stdout:start osd.0 2026-02-12T18:14:26.336 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec 2026-02-12T18:14:26.336 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:14:26.336 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:14:26.336 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:14:26.349 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:26.345+0000 7fe2b8e2e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:26.351 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:26.346+0000 7fe2b8e2e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:26.352 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:26.347+0000 7fe2b8e2e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:26.432 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T18:14:26.432 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:14:26.432 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T18:14:26.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:14:26.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:14:26.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:26.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:14:26.433 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:26.434 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:26.434 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:14:26.528 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:14:26.551 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:26.547+0000 7fe2b8e2e780 -1 Falling back to public interface 2026-02-12T18:14:26.891 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:26.887+0000 7fe2b8e2e780 -1 osd.0 0 log_to_monitors true 2026-02-12T18:14:27.529 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:14:27.529 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:27.529 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:14:27.530 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:14:27.530 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:27.530 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:14:27.625 INFO:tasks.workunit.client.0.trial116.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/4166141203,v1:127.0.0.1:6801/4166141203] [v2:127.0.0.1:6802/4166141203,v1:127.0.0.1:6803/4166141203] exists,up 6afddd7c-2049-4fc5-9aea-1d2b093a9399 2026-02-12T18:14:27.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:14:27.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:14:27.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:14:27.627 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-12T18:14:27.627 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T18:14:27.627 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T18:14:27.627 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:27.627 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:27.627 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:27.627 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-osd.0.asok 2026-02-12T18:14:27.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-12T18:14:27.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.19237/ceph-osd.0.asok log flush 2026-02-12T18:14:27.671 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:14:27.675 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-12T18:14:27.676 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-12T18:14:27.678 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-12T18:14:27.678 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:27.678 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:14:27.678 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-12T18:14:27.679 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:14:27.679 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:14:27.679 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:14:27.679 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:14:27.679 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:14:27.785 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:14:27.785 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:14:27.786 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:14:27.786 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:14:27.788 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:14:27.788 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:14:27.788 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:14:27.789 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:27.789 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:14:27.789 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:14:27.790 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:27.790 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:14:27.791 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:14:27.792 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-12T18:14:27.794 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:14:27.795 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:27.795 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:27.795 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19237 2026-02-12T18:14:27.796 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:14:27.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:14:27.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-12T18:14:27.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-12T18:14:27.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:27.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-12T18:14:27.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:27.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:14:27.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-12T18:14:27.797 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:14:27.797 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:14:27.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:14:27.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:14:27.798 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:14:27.798 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:14:27.798 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:14:27.799 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:14:27.799 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:14:27.800 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:14:27.800 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:14:27.800 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:14:27.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:27.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:14:27.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:14:27.801 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:27.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:14:27.803 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:14:27.803 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-12T18:14:27.804 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:14:27.804 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:27.804 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:27.804 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19237 2026-02-12T18:14:27.805 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:14:27.805 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:14:27.805 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-12T18:14:27.806 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T18:14:27.806 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:27.807 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:27.807 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19237 2026-02-12T18:14:27.807 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T18:14:27.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T18:14:27.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T18:14:27.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-12T18:14:27.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=lrc 2026-02-12T18:14:27.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:27.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T18:14:27.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T18:14:27.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T18:14:27.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-12T18:14:27.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=lrc 2026-02-12T18:14:27.826 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T18:14:27.826 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:27.826 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:27.826 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:27.826 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:27.826 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:27.826 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:27.826 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=lrc 2026-02-12T18:14:27.849 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T18:14:27.851 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T18:14:27.852 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:14:27.852 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:14:27.852 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T18:14:27.852 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T18:14:27.852 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:14:27.852 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:27.852 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:27.852 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:27.852 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:27.852 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:27.852 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:27.852 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:14:27.853 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19237/ceph-mon.a.asok config get fsid 2026-02-12T18:14:27.894 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T18:14:27.894 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:14:27.894 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:14:27.894 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T18:14:27.894 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T18:14:27.894 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:14:27.894 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:27.894 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:27.894 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:27.894 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:27.894 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:27.895 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:27.895 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:14:27.895 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19237/ceph-mon.a.asok config get mon_host 2026-02-12T18:14:27.936 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-12T18:14:27.936 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:27.936 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T18:14:27.936 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T18:14:27.936 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T18:14:27.936 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-12T18:14:27.937 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T18:14:28.034 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T18:14:28.034 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:28.035 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:28.035 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:28.035 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:28.035 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:28.035 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:28.036 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:14:28.036 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:14:28.051 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-12T18:14:28.051 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:28.051 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:28.051 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:28.052 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:28.052 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:28.052 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:28.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-12T18:14:28.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.19237/ceph-mon.a.asok log flush 2026-02-12T18:14:28.089 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:14:28.096 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=lrc 2026-02-12T18:14:28.096 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:28.096 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:14:28.096 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T18:14:28.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:14:28.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-12T18:14:28.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-12T18:14:28.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:14:28.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:14:28.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:14:28.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-12T18:14:28.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-12T18:14:28.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:14:28.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:14:28.098 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-12T18:14:28.098 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:14:28.098 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:28.098 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:28.098 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:28.098 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:28.098 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:28.098 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:28.098 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:28.098 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:14:28.098 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:14:28.098 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:14:28.099 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-12T18:14:28.099 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-12T18:14:28.099 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:14:28.099 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:14:28.099 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:14:28.099 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:14:28.099 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:14:28.099 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=lrc 2026-02-12T18:14:28.099 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-12T18:14:28.099 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:14:28.099 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f0467fce-faeb-4cf5-b1e1-ce2c2db515f2 2026-02-12T18:14:28.100 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 f0467fce-faeb-4cf5-b1e1-ce2c2db515f2' 2026-02-12T18:14:28.100 INFO:tasks.workunit.client.0.trial116.stdout:add osd0 f0467fce-faeb-4cf5-b1e1-ce2c2db515f2 2026-02-12T18:14:28.100 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:14:28.107 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCEGI5pQRgiBhAAqexJHMV9hxfFgIvfCYdadQ== 2026-02-12T18:14:28.107 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCEGI5pQRgiBhAAqexJHMV9hxfFgIvfCYdadQ=="}' 2026-02-12T18:14:28.107 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f0467fce-faeb-4cf5-b1e1-ce2c2db515f2 -i td/test-erasure-code-plugins/0/new.json 2026-02-12T18:14:28.208 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:28.214 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-12T18:14:28.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=lrc --mkfs --key AQCEGI5pQRgiBhAAqexJHMV9hxfFgIvfCYdadQ== --osd-uuid f0467fce-faeb-4cf5-b1e1-ce2c2db515f2 2026-02-12T18:14:28.231 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:28.226+0000 7f6b65a1a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:28.232 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:28.228+0000 7f6b65a1a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:28.233 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:28.229+0000 7f6b65a1a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:28.234 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:28.229+0000 7f6b65a1a780 -1 bdev(0x3eeac00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-12T18:14:28.234 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:28.229+0000 7f6b65a1a780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-12T18:14:28.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-12T18:14:28.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:14:28.639 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T18:14:28.639 INFO:tasks.workunit.client.0.trial116.stdout:adding osd0 key to auth repository 2026-02-12T18:14:28.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:14:28.739 INFO:tasks.workunit.client.0.trial116.stdout:start osd.0 2026-02-12T18:14:28.740 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T18:14:28.740 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=lrc 2026-02-12T18:14:28.740 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:14:28.740 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:14:28.741 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:14:28.754 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:28.749+0000 7f44b782e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:28.755 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:28.751+0000 7f44b782e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:28.756 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:28.752+0000 7f44b782e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:28.835 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T18:14:28.835 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:14:28.835 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T18:14:28.836 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:14:28.836 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:14:28.836 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:28.836 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:14:28.836 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:28.837 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:28.837 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:14:28.932 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:14:28.968 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:28.963+0000 7f44b782e780 -1 Falling back to public interface 2026-02-12T18:14:29.328 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:29.324+0000 7f44b782e780 -1 osd.0 0 log_to_monitors true 2026-02-12T18:14:29.933 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:14:29.933 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:29.934 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:14:29.934 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:14:29.935 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:29.935 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:14:30.030 INFO:tasks.workunit.client.0.trial116.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3605299727,v1:127.0.0.1:6801/3605299727] [v2:127.0.0.1:6802/3605299727,v1:127.0.0.1:6803/3605299727] exists,up f0467fce-faeb-4cf5-b1e1-ce2c2db515f2 2026-02-12T18:14:30.031 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:14:30.031 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:14:30.031 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:14:30.032 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-12T18:14:30.032 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T18:14:30.032 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T18:14:30.032 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:30.032 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:30.032 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:30.032 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-osd.0.asok 2026-02-12T18:14:30.032 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-12T18:14:30.032 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.19237/ceph-osd.0.asok log flush 2026-02-12T18:14:30.077 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:14:30.081 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-12T18:14:30.083 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-12T18:14:30.085 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-12T18:14:30.085 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:30.085 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:14:30.085 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-12T18:14:30.085 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:14:30.085 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:14:30.085 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:14:30.085 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:14:30.085 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:14:30.192 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:14:30.193 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:14:30.194 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:14:30.194 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:14:30.195 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:14:30.196 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:14:30.196 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:14:30.197 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:30.197 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:14:30.197 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:14:30.198 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:30.198 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:14:30.199 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:14:30.199 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-12T18:14:30.202 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:14:30.202 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:30.202 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:30.202 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19237 2026-02-12T18:14:30.204 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:14:30.204 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:14:30.204 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-12T18:14:30.204 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-12T18:14:30.204 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:30.204 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-12T18:14:30.205 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:30.205 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:14:30.205 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-12T18:14:30.205 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:14:30.205 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:14:30.205 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:14:30.205 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:14:30.205 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:14:30.206 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:14:30.206 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:14:30.206 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:14:30.206 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:14:30.207 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:14:30.207 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:14:30.208 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:14:30.208 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:30.208 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:14:30.209 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:14:30.209 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:30.209 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:14:30.210 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:14:30.210 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-12T18:14:30.211 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:14:30.212 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:30.212 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:30.212 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19237 2026-02-12T18:14:30.212 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:14:30.212 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:14:30.213 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-12T18:14:30.214 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T18:14:30.214 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:30.214 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:30.214 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19237 2026-02-12T18:14:30.215 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T18:14:30.216 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T18:14:30.216 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T18:14:30.216 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-12T18:14:30.216 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=isa 2026-02-12T18:14:30.216 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:30.216 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T18:14:30.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T18:14:30.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T18:14:30.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-12T18:14:30.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=isa 2026-02-12T18:14:30.233 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T18:14:30.233 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:30.233 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:30.234 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:30.234 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:30.234 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:30.234 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:30.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=isa 2026-02-12T18:14:30.256 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T18:14:30.257 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T18:14:30.257 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:14:30.258 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:14:30.258 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T18:14:30.258 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T18:14:30.258 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:14:30.258 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:30.258 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:30.258 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:30.258 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:30.258 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:30.258 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:30.259 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:14:30.259 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19237/ceph-mon.a.asok config get fsid 2026-02-12T18:14:30.299 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T18:14:30.300 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:14:30.300 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:14:30.300 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T18:14:30.300 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T18:14:30.301 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:14:30.301 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:30.301 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:30.301 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:30.302 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:30.302 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:30.302 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:30.302 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:14:30.302 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19237/ceph-mon.a.asok config get mon_host 2026-02-12T18:14:30.342 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-12T18:14:30.342 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:30.343 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T18:14:30.343 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T18:14:30.343 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T18:14:30.343 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-12T18:14:30.343 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T18:14:30.443 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T18:14:30.444 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:30.444 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:30.444 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:30.444 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:30.444 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:30.444 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:30.444 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:14:30.445 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:14:30.459 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-12T18:14:30.459 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:30.460 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:30.460 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:30.460 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:30.460 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:30.460 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:30.460 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-12T18:14:30.460 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.19237/ceph-mon.a.asok log flush 2026-02-12T18:14:30.497 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:14:30.504 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=isa 2026-02-12T18:14:30.504 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:30.504 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:14:30.504 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T18:14:30.504 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:14:30.504 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-12T18:14:30.505 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-12T18:14:30.505 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:14:30.505 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:14:30.505 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:14:30.505 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-12T18:14:30.505 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-12T18:14:30.505 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:14:30.505 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:14:30.505 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-12T18:14:30.505 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:14:30.505 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:30.505 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:30.506 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:30.506 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:30.506 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:30.506 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:30.506 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:30.506 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:14:30.506 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:14:30.506 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:14:30.506 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-12T18:14:30.506 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-12T18:14:30.506 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:14:30.506 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:14:30.506 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:14:30.507 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:14:30.507 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:14:30.507 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=isa 2026-02-12T18:14:30.507 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-12T18:14:30.507 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:14:30.507 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e9b3182d-1f1f-4eb8-be7e-5870746a187c 2026-02-12T18:14:30.507 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 e9b3182d-1f1f-4eb8-be7e-5870746a187c' 2026-02-12T18:14:30.507 INFO:tasks.workunit.client.0.trial116.stdout:add osd0 e9b3182d-1f1f-4eb8-be7e-5870746a187c 2026-02-12T18:14:30.508 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:14:30.513 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCGGI5pXZ9cHhAAlwVWcVUXwbCHQmVD4oPepQ== 2026-02-12T18:14:30.513 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCGGI5pXZ9cHhAAlwVWcVUXwbCHQmVD4oPepQ=="}' 2026-02-12T18:14:30.513 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e9b3182d-1f1f-4eb8-be7e-5870746a187c -i td/test-erasure-code-plugins/0/new.json 2026-02-12T18:14:30.614 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:30.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-12T18:14:30.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=isa --mkfs --key AQCGGI5pXZ9cHhAAlwVWcVUXwbCHQmVD4oPepQ== --osd-uuid e9b3182d-1f1f-4eb8-be7e-5870746a187c 2026-02-12T18:14:30.637 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:30.633+0000 7f12e8c4b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:30.639 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:30.634+0000 7f12e8c4b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:30.639 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:30.635+0000 7f12e8c4b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:30.640 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:30.635+0000 7f12e8c4b780 -1 bdev(0x3a66c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-12T18:14:30.640 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:30.635+0000 7f12e8c4b780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-12T18:14:31.026 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-12T18:14:31.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:14:31.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T18:14:31.028 INFO:tasks.workunit.client.0.trial116.stdout:adding osd0 key to auth repository 2026-02-12T18:14:31.028 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:14:31.131 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T18:14:31.131 INFO:tasks.workunit.client.0.trial116.stdout:start osd.0 2026-02-12T18:14:31.132 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=isa 2026-02-12T18:14:31.132 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:14:31.132 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:14:31.132 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:14:31.145 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:31.141+0000 7f32aea0d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:31.147 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:31.143+0000 7f32aea0d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:31.148 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:31.143+0000 7f32aea0d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:31.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T18:14:31.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:14:31.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T18:14:31.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:14:31.230 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:14:31.230 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:31.230 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:14:31.230 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:31.230 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:31.231 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:14:31.325 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:14:31.361 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:31.357+0000 7f32aea0d780 -1 Falling back to public interface 2026-02-12T18:14:31.696 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:31.691+0000 7f32aea0d780 -1 osd.0 0 log_to_monitors true 2026-02-12T18:14:32.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:14:32.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:32.327 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:14:32.327 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:14:32.327 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:32.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:14:32.426 INFO:tasks.workunit.client.0.trial116.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/118614949,v1:127.0.0.1:6801/118614949] [v2:127.0.0.1:6802/118614949,v1:127.0.0.1:6803/118614949] exists,up e9b3182d-1f1f-4eb8-be7e-5870746a187c 2026-02-12T18:14:32.427 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:14:32.427 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:14:32.428 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:14:32.428 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-12T18:14:32.428 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T18:14:32.428 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T18:14:32.428 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:32.428 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:32.428 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:32.428 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-osd.0.asok 2026-02-12T18:14:32.428 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-12T18:14:32.428 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.19237/ceph-osd.0.asok log flush 2026-02-12T18:14:32.472 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:14:32.475 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-12T18:14:32.477 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-12T18:14:32.479 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-12T18:14:32.479 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:32.479 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:14:32.479 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-12T18:14:32.479 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:14:32.479 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:14:32.480 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:14:32.480 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:14:32.480 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:14:32.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:14:32.586 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:14:32.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:14:32.587 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:14:32.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:14:32.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:14:32.589 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:14:32.590 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:32.590 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:14:32.590 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:14:32.591 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:32.592 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:14:32.592 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:14:32.592 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-12T18:14:32.595 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:14:32.595 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:32.596 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:32.596 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19237 2026-02-12T18:14:32.598 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:14:32.598 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:14:32.598 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:72: TEST_preload_no_warning: return 0 2026-02-12T18:14:32.598 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-12T18:14:32.598 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_preload_no_warning_default td/test-erasure-code-plugins 2026-02-12T18:14:32.598 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:76: TEST_preload_no_warning_default: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:32.598 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:78: TEST_preload_no_warning_default: setup td/test-erasure-code-plugins 2026-02-12T18:14:32.598 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:32.598 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-12T18:14:32.598 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:32.598 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:14:32.599 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-12T18:14:32.599 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:14:32.599 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:14:32.599 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:14:32.599 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:14:32.599 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:14:32.600 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:14:32.600 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:14:32.600 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:14:32.600 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:14:32.601 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:14:32.601 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:14:32.601 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:14:32.602 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:32.602 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:14:32.602 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:14:32.602 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:32.603 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:14:32.604 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:14:32.604 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-12T18:14:32.605 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:14:32.605 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:32.605 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:32.605 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19237 2026-02-12T18:14:32.606 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:14:32.606 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:14:32.606 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-12T18:14:32.607 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T18:14:32.607 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:32.607 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:32.607 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19237 2026-02-12T18:14:32.608 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T18:14:32.609 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T18:14:32.609 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T18:14:32.610 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-12T18:14:32.610 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:79: TEST_preload_no_warning_default: run_mon td/test-erasure-code-plugins a 2026-02-12T18:14:32.610 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:32.610 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T18:14:32.610 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T18:14:32.610 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T18:14:32.610 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-12T18:14:32.610 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins 2026-02-12T18:14:32.626 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T18:14:32.626 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:32.626 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:32.626 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:32.626 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:32.626 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:32.627 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:32.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T18:14:32.648 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T18:14:32.650 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T18:14:32.650 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:14:32.650 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:14:32.650 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T18:14:32.650 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T18:14:32.650 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:14:32.651 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:32.651 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:32.651 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:32.651 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:32.651 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:32.651 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:32.651 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:14:32.652 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19237/ceph-mon.a.asok config get fsid 2026-02-12T18:14:32.697 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T18:14:32.697 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:14:32.698 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:14:32.698 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T18:14:32.698 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T18:14:32.699 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:14:32.699 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:32.699 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:32.699 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:32.699 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:32.699 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:32.699 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:32.700 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:14:32.700 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19237/ceph-mon.a.asok config get mon_host 2026-02-12T18:14:32.743 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:80: TEST_preload_no_warning_default: get_asok_path mon.a 2026-02-12T18:14:32.744 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:32.744 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:32.744 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:32.744 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:32.744 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:32.744 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:32.745 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:80: TEST_preload_no_warning_default: CEPH_ARGS= 2026-02-12T18:14:32.745 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:80: TEST_preload_no_warning_default: ceph --admin-daemon /tmp/ceph-asok.19237/ceph-mon.a.asok log flush 2026-02-12T18:14:32.782 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:14:32.786 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:81: TEST_preload_no_warning_default: run_mgr td/test-erasure-code-plugins x 2026-02-12T18:14:32.786 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:32.786 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T18:14:32.786 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T18:14:32.786 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T18:14:32.786 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-12T18:14:32.786 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T18:14:32.885 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T18:14:32.885 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:32.885 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:32.885 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:32.885 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:32.885 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:32.885 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:32.886 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:14:32.886 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:14:32.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:82: TEST_preload_no_warning_default: run_osd td/test-erasure-code-plugins 0 2026-02-12T18:14:32.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:32.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:14:32.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T18:14:32.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:14:32.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-12T18:14:32.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-12T18:14:32.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:14:32.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:14:32.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:14:32.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-12T18:14:32.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-12T18:14:32.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:14:32.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:14:32.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-12T18:14:32.904 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:14:32.904 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:32.904 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:32.904 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:32.904 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:32.904 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:32.904 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:32.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:32.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:14:32.905 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:14:32.905 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:14:32.905 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-12T18:14:32.905 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-12T18:14:32.905 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:14:32.905 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:14:32.905 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:14:32.905 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:14:32.905 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:14:32.905 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:14:32.905 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-12T18:14:32.905 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:14:32.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7fb5ca7f-559c-4afe-a725-74b64e723e0a 2026-02-12T18:14:32.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 7fb5ca7f-559c-4afe-a725-74b64e723e0a' 2026-02-12T18:14:32.906 INFO:tasks.workunit.client.0.trial116.stdout:add osd0 7fb5ca7f-559c-4afe-a725-74b64e723e0a 2026-02-12T18:14:32.907 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:14:32.912 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCIGI5pgO0iNhAAw4mbpwOp2eeJ9aKFDDVVJQ== 2026-02-12T18:14:32.912 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCIGI5pgO0iNhAAw4mbpwOp2eeJ9aKFDDVVJQ=="}' 2026-02-12T18:14:32.912 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7fb5ca7f-559c-4afe-a725-74b64e723e0a -i td/test-erasure-code-plugins/0/new.json 2026-02-12T18:14:33.013 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:33.020 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-12T18:14:33.020 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCIGI5pgO0iNhAAw4mbpwOp2eeJ9aKFDDVVJQ== --osd-uuid 7fb5ca7f-559c-4afe-a725-74b64e723e0a 2026-02-12T18:14:33.036 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:33.031+0000 7f6c4c863780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:33.037 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:33.033+0000 7f6c4c863780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:33.038 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:33.034+0000 7f6c4c863780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:33.039 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:33.034+0000 7f6c4c863780 -1 bdev(0x2beac00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-12T18:14:33.039 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:33.035+0000 7f6c4c863780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-12T18:14:33.422 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-12T18:14:33.422 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:14:33.423 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T18:14:33.423 INFO:tasks.workunit.client.0.trial116.stdout:adding osd0 key to auth repository 2026-02-12T18:14:33.424 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:14:33.527 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T18:14:33.527 INFO:tasks.workunit.client.0.trial116.stdout:start osd.0 2026-02-12T18:14:33.528 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:14:33.528 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:14:33.528 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:14:33.528 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:14:33.541 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:33.536+0000 7ffa82467780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:33.542 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:33.538+0000 7ffa82467780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:33.543 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:33.538+0000 7ffa82467780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:33.625 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T18:14:33.625 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:14:33.625 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T18:14:33.625 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:14:33.625 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:14:33.625 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:33.625 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:14:33.626 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:33.626 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:33.626 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:14:33.720 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:14:33.725 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:33.721+0000 7ffa82467780 -1 Falling back to public interface 2026-02-12T18:14:34.057 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:34.052+0000 7ffa82467780 -1 osd.0 0 log_to_monitors true 2026-02-12T18:14:34.721 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:14:34.721 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:34.722 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:14:34.722 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:14:34.722 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:34.723 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:14:34.817 INFO:tasks.workunit.client.0.trial116.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/338367646,v1:127.0.0.1:6801/338367646] [v2:127.0.0.1:6802/338367646,v1:127.0.0.1:6803/338367646] exists,up 7fb5ca7f-559c-4afe-a725-74b64e723e0a 2026-02-12T18:14:34.819 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:14:34.819 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:14:34.819 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:14:34.819 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:83: TEST_preload_no_warning_default: get_asok_path osd.0 2026-02-12T18:14:34.819 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T18:14:34.819 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T18:14:34.820 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:34.820 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:34.820 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:34.820 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-osd.0.asok 2026-02-12T18:14:34.820 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:83: TEST_preload_no_warning_default: CEPH_ARGS= 2026-02-12T18:14:34.820 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:83: TEST_preload_no_warning_default: ceph --admin-daemon /tmp/ceph-asok.19237/ceph-osd.0.asok log flush 2026-02-12T18:14:34.865 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:14:34.869 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:84: TEST_preload_no_warning_default: grep 'WARNING: osd_erasure_code_plugins' td/test-erasure-code-plugins/mon.a.log 2026-02-12T18:14:34.871 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:85: TEST_preload_no_warning_default: grep 'WARNING: osd_erasure_code_plugins' td/test-erasure-code-plugins/osd.0.log 2026-02-12T18:14:34.873 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:86: TEST_preload_no_warning_default: teardown td/test-erasure-code-plugins 2026-02-12T18:14:34.873 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:34.873 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:14:34.873 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-12T18:14:34.873 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:14:34.874 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:14:34.874 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:14:34.874 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:14:34.874 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:14:34.980 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:14:34.981 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:14:34.982 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:14:34.982 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:14:34.983 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:14:34.983 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:14:34.984 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:14:34.984 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:34.985 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:14:34.985 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:14:34.985 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:34.986 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:14:34.986 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:14:34.987 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-12T18:14:34.990 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:14:34.990 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:34.990 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:34.990 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19237 2026-02-12T18:14:34.992 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:14:34.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:14:34.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:88: TEST_preload_no_warning_default: return 0 2026-02-12T18:14:34.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-12T18:14:34.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_preload_warning td/test-erasure-code-plugins 2026-02-12T18:14:34.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:41: TEST_preload_warning: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:34.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-12T18:14:34.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-12T18:14:34.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:34.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-12T18:14:34.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:34.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:14:34.994 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-12T18:14:34.994 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:14:34.994 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:14:34.994 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:14:34.994 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:14:34.994 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:14:34.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:14:34.995 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:14:34.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:14:34.995 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:14:34.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:14:34.996 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:14:34.996 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:14:34.996 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:34.996 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:14:34.997 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:14:34.997 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:34.997 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:14:34.998 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:14:34.998 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-12T18:14:34.999 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:14:34.999 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:34.999 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:34.999 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19237 2026-02-12T18:14:35.000 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:14:35.000 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:14:35.000 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-12T18:14:35.001 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T18:14:35.001 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:35.001 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:35.001 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19237 2026-02-12T18:14:35.002 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T18:14:35.003 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T18:14:35.003 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T18:14:35.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-12T18:14:35.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure_generic 2026-02-12T18:14:35.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:35.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T18:14:35.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T18:14:35.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T18:14:35.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-12T18:14:35.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure_generic 2026-02-12T18:14:35.020 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T18:14:35.021 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:35.021 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:35.021 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:35.021 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:35.021 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:35.021 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:35.021 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure_generic 2026-02-12T18:14:35.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T18:14:35.046 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T18:14:35.046 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:14:35.046 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:14:35.046 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T18:14:35.046 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T18:14:35.046 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:14:35.046 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:35.046 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:35.047 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:35.047 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:35.047 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:35.047 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:35.048 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:14:35.048 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19237/ceph-mon.a.asok config get fsid 2026-02-12T18:14:35.092 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T18:14:35.092 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:14:35.092 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:14:35.092 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T18:14:35.092 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T18:14:35.092 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:14:35.092 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:35.092 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:35.093 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:35.093 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:35.093 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:35.093 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:35.093 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:14:35.093 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19237/ceph-mon.a.asok config get mon_host 2026-02-12T18:14:35.133 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-12T18:14:35.133 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:35.133 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T18:14:35.133 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T18:14:35.133 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T18:14:35.136 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-12T18:14:35.136 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T18:14:35.228 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T18:14:35.228 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:35.228 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:35.228 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:35.228 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:35.228 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:35.228 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:35.229 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:14:35.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:14:35.245 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-12T18:14:35.245 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:35.245 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:35.245 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:35.245 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:35.245 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:35.246 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:35.246 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-12T18:14:35.246 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.19237/ceph-mon.a.asok log flush 2026-02-12T18:14:35.286 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:14:35.295 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure_generic 2026-02-12T18:14:35.295 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:35.295 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:14:35.295 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T18:14:35.295 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:14:35.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-12T18:14:35.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-12T18:14:35.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:14:35.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:14:35.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:14:35.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-12T18:14:35.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-12T18:14:35.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:14:35.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:14:35.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-12T18:14:35.296 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:14:35.297 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:35.297 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:35.297 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:35.297 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:35.297 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:35.297 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:35.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:35.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:14:35.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:14:35.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:14:35.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-12T18:14:35.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-12T18:14:35.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:14:35.298 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:14:35.298 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:14:35.298 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:14:35.298 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:14:35.298 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure_generic 2026-02-12T18:14:35.298 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-12T18:14:35.298 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:14:35.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=838dacad-6d56-4e98-b13a-62162ff1b70f 2026-02-12T18:14:35.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 838dacad-6d56-4e98-b13a-62162ff1b70f' 2026-02-12T18:14:35.299 INFO:tasks.workunit.client.0.trial116.stdout:add osd0 838dacad-6d56-4e98-b13a-62162ff1b70f 2026-02-12T18:14:35.299 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:14:35.306 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCLGI5pecn9ERAA1EJEjDSE8cHB2qCcBhMytA== 2026-02-12T18:14:35.306 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCLGI5pecn9ERAA1EJEjDSE8cHB2qCcBhMytA=="}' 2026-02-12T18:14:35.306 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 838dacad-6d56-4e98-b13a-62162ff1b70f -i td/test-erasure-code-plugins/0/new.json 2026-02-12T18:14:35.405 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:35.411 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-12T18:14:35.412 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_generic --mkfs --key AQCLGI5pecn9ERAA1EJEjDSE8cHB2qCcBhMytA== --osd-uuid 838dacad-6d56-4e98-b13a-62162ff1b70f 2026-02-12T18:14:35.427 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:35.422+0000 7fe43f621780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:35.428 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:35.424+0000 7fe43f621780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:35.429 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:35.425+0000 7fe43f621780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:35.430 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:35.425+0000 7fe43f621780 -1 bdev(0x3ad6c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-12T18:14:35.430 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:35.425+0000 7fe43f621780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-12T18:14:35.816 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-12T18:14:35.816 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:14:35.817 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T18:14:35.817 INFO:tasks.workunit.client.0.trial116.stdout:adding osd0 key to auth repository 2026-02-12T18:14:35.817 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:14:35.916 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T18:14:35.917 INFO:tasks.workunit.client.0.trial116.stdout:start osd.0 2026-02-12T18:14:35.918 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_generic 2026-02-12T18:14:35.918 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:14:35.918 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:14:35.918 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:14:35.932 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:35.927+0000 7f8986a65780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:35.933 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:35.929+0000 7f8986a65780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:35.934 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:35.930+0000 7f8986a65780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:36.014 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T18:14:36.014 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:14:36.014 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T18:14:36.014 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:14:36.014 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:14:36.014 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:36.014 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:14:36.015 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:36.015 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:36.015 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:14:36.108 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:14:36.124 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:36.119+0000 7f8986a65780 -1 Falling back to public interface 2026-02-12T18:14:36.457 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:36.453+0000 7f8986a65780 -1 osd.0 0 log_to_monitors true 2026-02-12T18:14:37.109 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:14:37.109 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:37.110 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:14:37.110 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:14:37.110 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:37.110 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:14:37.208 INFO:tasks.workunit.client.0.trial116.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/411079315,v1:127.0.0.1:6801/411079315] [v2:127.0.0.1:6802/411079315,v1:127.0.0.1:6803/411079315] exists,up 838dacad-6d56-4e98-b13a-62162ff1b70f 2026-02-12T18:14:37.210 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:14:37.211 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:14:37.211 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:14:37.211 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-12T18:14:37.211 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T18:14:37.211 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T18:14:37.211 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:37.211 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:37.211 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:37.211 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-osd.0.asok 2026-02-12T18:14:37.211 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-12T18:14:37.211 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.19237/ceph-osd.0.asok log flush 2026-02-12T18:14:37.254 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:14:37.258 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-12T18:14:37.260 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:35.032+0000 7fbc4c211c00 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-12T18:14:37.260 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_generic' td/test-erasure-code-plugins/osd.0.log 2026-02-12T18:14:37.261 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:36.120+0000 7f8986a65780 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-12T18:14:37.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-12T18:14:37.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:37.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:14:37.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-12T18:14:37.262 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:14:37.262 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:14:37.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:14:37.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:14:37.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:14:37.368 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:14:37.369 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:14:37.370 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:14:37.370 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:14:37.371 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:14:37.371 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:14:37.372 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:14:37.373 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:37.373 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:14:37.374 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:14:37.374 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:37.374 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:14:37.375 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:14:37.375 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-12T18:14:37.378 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:14:37.378 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:37.378 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:37.378 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19237 2026-02-12T18:14:37.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:14:37.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:14:37.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-12T18:14:37.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-12T18:14:37.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:37.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-12T18:14:37.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:37.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:14:37.382 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-12T18:14:37.382 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:14:37.382 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:14:37.382 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:14:37.382 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:14:37.382 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:14:37.383 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:14:37.383 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:14:37.383 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:14:37.383 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:14:37.384 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:14:37.384 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:14:37.384 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:14:37.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:37.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:14:37.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:14:37.385 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:37.386 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:14:37.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:14:37.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-12T18:14:37.388 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:14:37.388 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:37.388 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:37.388 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19237 2026-02-12T18:14:37.389 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:14:37.389 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:14:37.389 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-12T18:14:37.390 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T18:14:37.390 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:37.390 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:37.390 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19237 2026-02-12T18:14:37.391 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T18:14:37.392 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T18:14:37.392 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T18:14:37.392 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-12T18:14:37.392 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure_sse3 2026-02-12T18:14:37.393 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:37.393 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T18:14:37.393 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T18:14:37.393 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T18:14:37.393 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-12T18:14:37.393 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure_sse3 2026-02-12T18:14:37.410 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T18:14:37.410 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:37.410 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:37.410 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:37.410 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:37.410 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:37.411 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:37.411 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure_sse3 2026-02-12T18:14:37.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T18:14:37.434 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T18:14:37.434 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:14:37.435 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:14:37.435 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T18:14:37.435 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T18:14:37.436 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:14:37.436 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:37.436 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:37.436 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:37.436 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:37.436 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:37.436 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:37.437 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:14:37.437 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19237/ceph-mon.a.asok config get fsid 2026-02-12T18:14:37.478 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T18:14:37.478 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:14:37.478 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:14:37.478 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T18:14:37.479 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T18:14:37.479 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:14:37.479 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:37.479 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:37.479 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:37.479 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:37.479 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:37.479 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:37.480 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:14:37.480 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19237/ceph-mon.a.asok config get mon_host 2026-02-12T18:14:37.520 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-12T18:14:37.520 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:37.520 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T18:14:37.520 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T18:14:37.520 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T18:14:37.520 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-12T18:14:37.520 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T18:14:37.622 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T18:14:37.622 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:37.622 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:37.622 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:37.622 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:37.623 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:37.623 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:37.623 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:14:37.623 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:14:37.638 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-12T18:14:37.638 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:37.638 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:37.639 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:37.639 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:37.639 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:37.639 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:37.639 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-12T18:14:37.639 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.19237/ceph-mon.a.asok log flush 2026-02-12T18:14:37.678 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:14:37.685 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure_sse3 2026-02-12T18:14:37.685 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:37.685 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:14:37.685 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T18:14:37.685 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:14:37.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-12T18:14:37.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-12T18:14:37.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:14:37.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:14:37.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:14:37.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-12T18:14:37.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-12T18:14:37.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:14:37.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:14:37.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-12T18:14:37.686 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:14:37.686 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:37.687 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:37.687 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:37.687 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:37.687 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:37.687 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:37.687 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:37.687 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:14:37.687 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:14:37.687 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:14:37.687 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-12T18:14:37.687 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-12T18:14:37.687 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:14:37.687 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:14:37.688 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:14:37.688 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:14:37.688 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:14:37.688 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure_sse3 2026-02-12T18:14:37.688 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-12T18:14:37.688 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:14:37.688 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b6eb3768-47c3-4f63-9d50-c4bd0a1a2dc8 2026-02-12T18:14:37.688 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 b6eb3768-47c3-4f63-9d50-c4bd0a1a2dc8' 2026-02-12T18:14:37.688 INFO:tasks.workunit.client.0.trial116.stdout:add osd0 b6eb3768-47c3-4f63-9d50-c4bd0a1a2dc8 2026-02-12T18:14:37.689 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:14:37.695 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCNGI5pGpEwKRAAbzFviIaLKxMU6kPsSg/Ifw== 2026-02-12T18:14:37.695 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCNGI5pGpEwKRAAbzFviIaLKxMU6kPsSg/Ifw=="}' 2026-02-12T18:14:37.695 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b6eb3768-47c3-4f63-9d50-c4bd0a1a2dc8 -i td/test-erasure-code-plugins/0/new.json 2026-02-12T18:14:37.796 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:37.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-12T18:14:37.803 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse3 --mkfs --key AQCNGI5pGpEwKRAAbzFviIaLKxMU6kPsSg/Ifw== --osd-uuid b6eb3768-47c3-4f63-9d50-c4bd0a1a2dc8 2026-02-12T18:14:37.818 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:37.813+0000 7ff4b2c0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:37.819 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:37.815+0000 7ff4b2c0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:37.820 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:37.815+0000 7ff4b2c0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:37.820 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:37.816+0000 7ff4b2c0f780 -1 bdev(0x24dac00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-12T18:14:37.820 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:37.816+0000 7ff4b2c0f780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-12T18:14:38.221 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-12T18:14:38.222 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:14:38.222 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T18:14:38.223 INFO:tasks.workunit.client.0.trial116.stdout:adding osd0 key to auth repository 2026-02-12T18:14:38.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:14:38.325 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T18:14:38.325 INFO:tasks.workunit.client.0.trial116.stdout:start osd.0 2026-02-12T18:14:38.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse3 2026-02-12T18:14:38.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:14:38.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:14:38.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:14:38.339 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:38.335+0000 7fa1079fb780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:38.341 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:38.336+0000 7fa1079fb780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:38.342 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:38.337+0000 7fa1079fb780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:38.422 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T18:14:38.422 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:14:38.423 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T18:14:38.423 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:14:38.423 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:14:38.423 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:38.423 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:14:38.423 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:38.424 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:38.424 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:14:38.518 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:14:38.564 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:38.560+0000 7fa1079fb780 -1 Falling back to public interface 2026-02-12T18:14:38.886 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:38.882+0000 7fa1079fb780 -1 osd.0 0 log_to_monitors true 2026-02-12T18:14:39.519 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:14:39.519 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:39.520 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:14:39.520 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:14:39.520 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:39.521 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:14:39.616 INFO:tasks.workunit.client.0.trial116.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3631442488,v1:127.0.0.1:6801/3631442488] [v2:127.0.0.1:6802/3631442488,v1:127.0.0.1:6803/3631442488] exists,up b6eb3768-47c3-4f63-9d50-c4bd0a1a2dc8 2026-02-12T18:14:39.617 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:14:39.617 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:14:39.618 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:14:39.618 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-12T18:14:39.618 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T18:14:39.618 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T18:14:39.618 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:39.618 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:39.618 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:39.618 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-osd.0.asok 2026-02-12T18:14:39.618 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-12T18:14:39.618 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.19237/ceph-osd.0.asok log flush 2026-02-12T18:14:39.662 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:14:39.666 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-12T18:14:39.668 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:37.422+0000 7f937dc31c00 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-12T18:14:39.668 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3' td/test-erasure-code-plugins/osd.0.log 2026-02-12T18:14:39.669 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:38.560+0000 7fa1079fb780 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-12T18:14:39.670 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-12T18:14:39.670 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:39.670 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:14:39.671 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-12T18:14:39.671 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:14:39.671 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:14:39.671 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:14:39.671 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:14:39.671 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:14:39.776 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:14:39.776 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:14:39.777 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:14:39.778 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:14:39.779 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:14:39.779 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:14:39.779 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:14:39.780 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:39.780 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:14:39.781 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:14:39.781 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:39.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:14:39.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:14:39.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-12T18:14:39.786 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:14:39.786 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:39.786 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:39.786 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19237 2026-02-12T18:14:39.787 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:14:39.787 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:14:39.787 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-12T18:14:39.788 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-12T18:14:39.788 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:39.788 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-12T18:14:39.788 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:39.788 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:14:39.788 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-12T18:14:39.788 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:14:39.788 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:14:39.788 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:14:39.788 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:14:39.788 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:14:39.790 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:14:39.790 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:14:39.791 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:14:39.791 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:14:39.792 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:14:39.792 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:14:39.792 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:14:39.793 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:39.793 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:14:39.793 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:14:39.794 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:39.794 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:14:39.795 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:14:39.795 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-12T18:14:39.796 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:14:39.796 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:39.796 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:39.796 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19237 2026-02-12T18:14:39.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:14:39.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:14:39.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-12T18:14:39.798 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T18:14:39.798 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:39.799 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:39.799 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19237 2026-02-12T18:14:39.800 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T18:14:39.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T18:14:39.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T18:14:39.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-12T18:14:39.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure_sse4 2026-02-12T18:14:39.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:39.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T18:14:39.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T18:14:39.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T18:14:39.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-12T18:14:39.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure_sse4 2026-02-12T18:14:39.819 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T18:14:39.819 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:39.819 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:39.819 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:39.819 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:39.819 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:39.819 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:39.820 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure_sse4 2026-02-12T18:14:39.842 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T18:14:39.843 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T18:14:39.843 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:14:39.843 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:14:39.843 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T18:14:39.844 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T18:14:39.844 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:14:39.844 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:39.845 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:39.845 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:39.845 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:39.845 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:39.845 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:39.845 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:14:39.845 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19237/ceph-mon.a.asok config get fsid 2026-02-12T18:14:39.885 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T18:14:39.885 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:14:39.885 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:14:39.885 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T18:14:39.885 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T18:14:39.885 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:14:39.885 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:39.886 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:39.886 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:39.886 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:39.886 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:39.886 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:39.886 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:14:39.886 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19237/ceph-mon.a.asok config get mon_host 2026-02-12T18:14:39.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-12T18:14:39.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:39.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T18:14:39.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T18:14:39.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T18:14:39.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-12T18:14:39.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T18:14:40.027 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T18:14:40.027 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:40.028 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:40.028 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:40.028 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:40.028 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:40.028 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:40.028 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:14:40.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:14:40.044 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-12T18:14:40.044 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:40.044 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:40.044 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:40.044 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:40.044 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:40.044 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:40.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-12T18:14:40.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.19237/ceph-mon.a.asok log flush 2026-02-12T18:14:40.082 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:14:40.089 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure_sse4 2026-02-12T18:14:40.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:40.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:14:40.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T18:14:40.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:14:40.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-12T18:14:40.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-12T18:14:40.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:14:40.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:14:40.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:14:40.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-12T18:14:40.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-12T18:14:40.091 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:14:40.091 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:14:40.091 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-12T18:14:40.091 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:14:40.091 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:40.091 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:40.091 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:40.091 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:40.091 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:40.091 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:40.091 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:40.091 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:14:40.092 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:14:40.092 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:14:40.092 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-12T18:14:40.092 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-12T18:14:40.092 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:14:40.092 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:14:40.092 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:14:40.092 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:14:40.092 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:14:40.092 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure_sse4 2026-02-12T18:14:40.092 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-12T18:14:40.092 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:14:40.092 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3ad95d32-0229-4257-abad-95753762d839 2026-02-12T18:14:40.093 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 3ad95d32-0229-4257-abad-95753762d839' 2026-02-12T18:14:40.093 INFO:tasks.workunit.client.0.trial116.stdout:add osd0 3ad95d32-0229-4257-abad-95753762d839 2026-02-12T18:14:40.093 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:14:40.098 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCQGI5prkamBRAAycmHWbDL4LLBD2Fms07RPw== 2026-02-12T18:14:40.099 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCQGI5prkamBRAAycmHWbDL4LLBD2Fms07RPw=="}' 2026-02-12T18:14:40.099 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3ad95d32-0229-4257-abad-95753762d839 -i td/test-erasure-code-plugins/0/new.json 2026-02-12T18:14:40.198 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:40.204 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-12T18:14:40.205 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse4 --mkfs --key AQCQGI5prkamBRAAycmHWbDL4LLBD2Fms07RPw== --osd-uuid 3ad95d32-0229-4257-abad-95753762d839 2026-02-12T18:14:40.220 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:40.215+0000 7fcd2d634780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:40.221 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:40.217+0000 7fcd2d634780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:40.222 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:40.217+0000 7fcd2d634780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:40.222 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:40.218+0000 7fcd2d634780 -1 bdev(0x3476c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-12T18:14:40.222 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:40.218+0000 7fcd2d634780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-12T18:14:40.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-12T18:14:40.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:14:40.637 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T18:14:40.637 INFO:tasks.workunit.client.0.trial116.stdout:adding osd0 key to auth repository 2026-02-12T18:14:40.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:14:40.740 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T18:14:40.740 INFO:tasks.workunit.client.0.trial116.stdout:start osd.0 2026-02-12T18:14:40.741 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse4 2026-02-12T18:14:40.741 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:14:40.741 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:14:40.741 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:14:40.754 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:40.749+0000 7f6640437780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:40.755 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:40.751+0000 7f6640437780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:40.756 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:40.751+0000 7f6640437780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:40.840 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T18:14:40.841 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:14:40.841 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T18:14:40.841 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:14:40.841 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:14:40.841 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:40.841 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:14:40.841 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:40.842 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:40.842 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:14:40.935 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:14:40.967 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:40.962+0000 7f6640437780 -1 Falling back to public interface 2026-02-12T18:14:41.274 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:41.269+0000 7f6640437780 -1 osd.0 0 log_to_monitors true 2026-02-12T18:14:41.937 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:14:41.937 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:41.937 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:14:41.938 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:14:41.938 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:41.938 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:14:42.034 INFO:tasks.workunit.client.0.trial116.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/454683871,v1:127.0.0.1:6801/454683871] [v2:127.0.0.1:6802/454683871,v1:127.0.0.1:6803/454683871] exists,up 3ad95d32-0229-4257-abad-95753762d839 2026-02-12T18:14:42.035 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:14:42.036 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:14:42.036 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:14:42.036 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-12T18:14:42.036 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T18:14:42.036 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T18:14:42.036 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:42.036 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:42.036 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:42.036 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-osd.0.asok 2026-02-12T18:14:42.036 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-12T18:14:42.037 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.19237/ceph-osd.0.asok log flush 2026-02-12T18:14:42.081 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:14:42.085 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-12T18:14:42.086 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:39.829+0000 7fcae865bc00 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-12T18:14:42.087 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4' td/test-erasure-code-plugins/osd.0.log 2026-02-12T18:14:42.088 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:40.963+0000 7f6640437780 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-12T18:14:42.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-12T18:14:42.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:42.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:14:42.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-12T18:14:42.090 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:14:42.091 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:14:42.091 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:14:42.091 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:14:42.091 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:14:42.195 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:14:42.196 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:14:42.197 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:14:42.197 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:14:42.198 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:14:42.198 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:14:42.199 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:14:42.200 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:42.200 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:14:42.200 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:14:42.200 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:42.201 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:14:42.202 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:14:42.202 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-12T18:14:42.205 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:14:42.205 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:42.205 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:42.205 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19237 2026-02-12T18:14:42.208 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:14:42.208 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:14:42.208 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-12T18:14:42.208 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-12T18:14:42.208 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:42.208 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-12T18:14:42.208 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:42.209 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:14:42.209 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-12T18:14:42.209 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:14:42.209 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:14:42.209 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:14:42.209 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:14:42.209 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:14:42.211 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:14:42.211 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:14:42.211 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:14:42.211 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:14:42.211 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:14:42.211 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:14:42.212 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:14:42.212 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:42.213 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:14:42.213 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:14:42.213 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:42.213 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:14:42.214 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:14:42.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-12T18:14:42.215 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:14:42.215 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:42.216 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:42.216 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19237 2026-02-12T18:14:42.216 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:14:42.216 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:14:42.216 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-12T18:14:42.217 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T18:14:42.217 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:42.218 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:42.218 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19237 2026-02-12T18:14:42.219 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T18:14:42.220 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T18:14:42.220 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T18:14:42.220 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-12T18:14:42.220 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec_generic 2026-02-12T18:14:42.220 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:42.220 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T18:14:42.220 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T18:14:42.220 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T18:14:42.220 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-12T18:14:42.221 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec_generic 2026-02-12T18:14:42.237 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T18:14:42.237 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:42.237 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:42.237 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:42.237 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:42.237 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:42.237 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:42.238 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec_generic 2026-02-12T18:14:42.259 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T18:14:42.260 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T18:14:42.260 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:14:42.261 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:14:42.261 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T18:14:42.261 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T18:14:42.261 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:14:42.261 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:42.261 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:42.262 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:42.262 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:42.262 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:42.262 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:42.262 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:14:42.263 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19237/ceph-mon.a.asok config get fsid 2026-02-12T18:14:42.303 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T18:14:42.303 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:14:42.303 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:14:42.304 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T18:14:42.304 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T18:14:42.305 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:14:42.305 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:42.305 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:42.305 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:42.306 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:42.306 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:42.306 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:42.306 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:14:42.306 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19237/ceph-mon.a.asok config get mon_host 2026-02-12T18:14:42.347 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-12T18:14:42.347 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:42.347 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T18:14:42.347 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T18:14:42.347 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T18:14:42.347 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-12T18:14:42.347 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T18:14:42.444 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T18:14:42.445 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:42.445 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:42.445 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:42.445 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:42.445 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:42.445 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:42.446 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:14:42.446 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:14:42.461 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-12T18:14:42.462 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:42.462 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:42.462 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:42.462 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:42.462 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:42.462 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:42.462 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-12T18:14:42.462 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.19237/ceph-mon.a.asok log flush 2026-02-12T18:14:42.499 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:14:42.506 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec_generic 2026-02-12T18:14:42.506 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:42.506 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:14:42.506 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T18:14:42.506 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:14:42.506 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-12T18:14:42.506 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-12T18:14:42.506 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:14:42.506 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:14:42.507 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:14:42.507 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-12T18:14:42.507 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-12T18:14:42.507 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:14:42.507 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:14:42.507 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-12T18:14:42.507 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:14:42.507 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:42.507 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:42.507 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:42.507 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:42.507 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:42.508 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:42.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:42.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:14:42.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:14:42.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:14:42.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-12T18:14:42.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-12T18:14:42.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:14:42.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:14:42.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:14:42.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:14:42.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:14:42.509 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec_generic 2026-02-12T18:14:42.509 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-12T18:14:42.509 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:14:42.509 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5d5bafe9-5ef1-4d15-9b87-27feb61f9d5e 2026-02-12T18:14:42.509 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 5d5bafe9-5ef1-4d15-9b87-27feb61f9d5e' 2026-02-12T18:14:42.510 INFO:tasks.workunit.client.0.trial116.stdout:add osd0 5d5bafe9-5ef1-4d15-9b87-27feb61f9d5e 2026-02-12T18:14:42.510 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:14:42.516 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCSGI5pv+qOHhAAP8DNBs9FBcZilbT70mNAbg== 2026-02-12T18:14:42.516 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCSGI5pv+qOHhAAP8DNBs9FBcZilbT70mNAbg=="}' 2026-02-12T18:14:42.517 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5d5bafe9-5ef1-4d15-9b87-27feb61f9d5e -i td/test-erasure-code-plugins/0/new.json 2026-02-12T18:14:42.618 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:42.625 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-12T18:14:42.626 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_generic --mkfs --key AQCSGI5pv+qOHhAAP8DNBs9FBcZilbT70mNAbg== --osd-uuid 5d5bafe9-5ef1-4d15-9b87-27feb61f9d5e 2026-02-12T18:14:42.641 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:42.637+0000 7f8b6ec2f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:42.642 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:42.638+0000 7f8b6ec2f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:42.643 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:42.639+0000 7f8b6ec2f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:42.643 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:42.639+0000 7f8b6ec2f780 -1 bdev(0x2e2cc00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-12T18:14:42.643 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:42.639+0000 7f8b6ec2f780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-12T18:14:43.062 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-12T18:14:43.062 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:14:43.062 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T18:14:43.063 INFO:tasks.workunit.client.0.trial116.stdout:adding osd0 key to auth repository 2026-02-12T18:14:43.063 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:14:43.163 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T18:14:43.163 INFO:tasks.workunit.client.0.trial116.stdout:start osd.0 2026-02-12T18:14:43.164 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_generic 2026-02-12T18:14:43.164 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:14:43.164 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:14:43.164 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:14:43.177 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:43.173+0000 7faf8880d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:43.179 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:43.174+0000 7faf8880d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:43.180 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:43.175+0000 7faf8880d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:43.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T18:14:43.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:14:43.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T18:14:43.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:14:43.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:14:43.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:43.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:14:43.263 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:43.264 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:43.264 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:14:43.358 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:14:43.399 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:43.395+0000 7faf8880d780 -1 Falling back to public interface 2026-02-12T18:14:43.838 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:43.834+0000 7faf8880d780 -1 osd.0 0 log_to_monitors true 2026-02-12T18:14:44.359 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:14:44.360 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:44.360 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:14:44.361 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:14:44.361 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:44.361 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:14:44.460 INFO:tasks.workunit.client.0.trial116.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1951011434,v1:127.0.0.1:6801/1951011434] [v2:127.0.0.1:6802/1951011434,v1:127.0.0.1:6803/1951011434] exists,up 5d5bafe9-5ef1-4d15-9b87-27feb61f9d5e 2026-02-12T18:14:44.462 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:14:44.462 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:14:44.462 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:14:44.462 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-12T18:14:44.462 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T18:14:44.462 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T18:14:44.462 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:44.462 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:44.463 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:44.463 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-osd.0.asok 2026-02-12T18:14:44.463 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-12T18:14:44.463 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.19237/ceph-osd.0.asok log flush 2026-02-12T18:14:44.506 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:14:44.509 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-12T18:14:44.511 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:42.247+0000 7eff8d243c00 0 WARNING: osd_erasure_code_plugins contains plugin shec_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-12T18:14:44.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_generic' td/test-erasure-code-plugins/osd.0.log 2026-02-12T18:14:44.512 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:43.395+0000 7faf8880d780 0 WARNING: osd_erasure_code_plugins contains plugin shec_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-12T18:14:44.513 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-12T18:14:44.513 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:44.513 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:14:44.513 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-12T18:14:44.514 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:14:44.514 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:14:44.514 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:14:44.514 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:14:44.514 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:14:44.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:14:44.620 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:14:44.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:14:44.621 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:14:44.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:14:44.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:14:44.623 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:14:44.624 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:44.624 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:14:44.624 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:14:44.625 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:44.625 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:14:44.626 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:14:44.626 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-12T18:14:44.629 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:14:44.629 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:44.629 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:44.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19237 2026-02-12T18:14:44.631 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:14:44.632 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:14:44.632 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-12T18:14:44.632 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-12T18:14:44.632 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:44.632 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-12T18:14:44.632 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:44.632 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:14:44.632 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-12T18:14:44.632 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:14:44.632 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:14:44.633 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:14:44.633 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:14:44.633 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:14:44.634 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:14:44.634 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:14:44.634 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:14:44.634 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:14:44.634 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:14:44.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:14:44.635 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:14:44.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:44.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:14:44.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:14:44.636 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:44.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:14:44.637 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:14:44.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-12T18:14:44.639 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:14:44.639 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:44.639 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:44.639 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19237 2026-02-12T18:14:44.639 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:14:44.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:14:44.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-12T18:14:44.641 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T18:14:44.641 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:44.641 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:44.641 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19237 2026-02-12T18:14:44.642 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T18:14:44.643 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T18:14:44.643 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T18:14:44.644 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-12T18:14:44.644 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec_sse3 2026-02-12T18:14:44.644 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:44.644 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T18:14:44.644 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T18:14:44.644 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T18:14:44.644 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-12T18:14:44.644 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec_sse3 2026-02-12T18:14:44.660 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T18:14:44.660 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:44.660 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:44.661 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:44.661 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:44.661 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:44.661 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:44.661 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec_sse3 2026-02-12T18:14:44.685 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T18:14:44.687 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T18:14:44.687 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:14:44.687 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:14:44.687 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T18:14:44.687 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T18:14:44.687 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:14:44.687 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:44.688 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:44.688 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:44.688 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:44.688 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:44.688 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:44.688 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:14:44.688 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19237/ceph-mon.a.asok config get fsid 2026-02-12T18:14:44.728 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T18:14:44.729 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:14:44.729 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:14:44.729 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T18:14:44.729 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T18:14:44.730 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:14:44.730 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:44.730 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:44.730 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:44.730 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:44.731 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:44.731 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:44.731 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:14:44.731 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19237/ceph-mon.a.asok config get mon_host 2026-02-12T18:14:44.776 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-12T18:14:44.776 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:44.777 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T18:14:44.777 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T18:14:44.777 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T18:14:44.777 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-12T18:14:44.777 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T18:14:44.877 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T18:14:44.877 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:44.877 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:44.877 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:44.877 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:44.878 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:44.878 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:44.878 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:14:44.878 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:14:44.893 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-12T18:14:44.893 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:44.893 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:44.893 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:44.893 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:44.893 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:44.893 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:44.893 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-12T18:14:44.894 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.19237/ceph-mon.a.asok log flush 2026-02-12T18:14:44.931 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:14:44.937 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec_sse3 2026-02-12T18:14:44.938 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:44.938 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:14:44.938 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T18:14:44.938 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:14:44.938 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-12T18:14:44.938 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-12T18:14:44.938 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:14:44.938 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:14:44.938 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:14:44.938 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-12T18:14:44.939 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-12T18:14:44.939 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:14:44.939 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:14:44.939 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-12T18:14:44.939 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:14:44.939 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:44.939 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:44.939 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:44.939 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:44.939 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:44.939 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:44.939 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:44.939 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:14:44.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:14:44.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:14:44.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-12T18:14:44.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-12T18:14:44.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:14:44.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:14:44.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:14:44.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:14:44.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:14:44.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec_sse3 2026-02-12T18:14:44.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-12T18:14:44.940 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:14:44.941 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=76d49fc6-66bd-490e-b703-6e6cd79a5fbd 2026-02-12T18:14:44.941 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 76d49fc6-66bd-490e-b703-6e6cd79a5fbd' 2026-02-12T18:14:44.941 INFO:tasks.workunit.client.0.trial116.stdout:add osd0 76d49fc6-66bd-490e-b703-6e6cd79a5fbd 2026-02-12T18:14:44.942 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:14:44.947 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCUGI5p+h1AOBAAEqqB8taDA7GCBvMxR10Z1w== 2026-02-12T18:14:44.947 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCUGI5p+h1AOBAAEqqB8taDA7GCBvMxR10Z1w=="}' 2026-02-12T18:14:44.947 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 76d49fc6-66bd-490e-b703-6e6cd79a5fbd -i td/test-erasure-code-plugins/0/new.json 2026-02-12T18:14:45.048 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:45.054 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-12T18:14:45.054 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse3 --mkfs --key AQCUGI5p+h1AOBAAEqqB8taDA7GCBvMxR10Z1w== --osd-uuid 76d49fc6-66bd-490e-b703-6e6cd79a5fbd 2026-02-12T18:14:45.070 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:45.066+0000 7f125b636780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:45.072 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:45.068+0000 7f125b636780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:45.073 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:45.069+0000 7f125b636780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:45.073 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:45.069+0000 7f125b636780 -1 bdev(0x3094c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-12T18:14:45.074 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:45.069+0000 7f125b636780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-12T18:14:45.482 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-12T18:14:45.482 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:14:45.483 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T18:14:45.483 INFO:tasks.workunit.client.0.trial116.stdout:adding osd0 key to auth repository 2026-02-12T18:14:45.484 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:14:45.584 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T18:14:45.584 INFO:tasks.workunit.client.0.trial116.stdout:start osd.0 2026-02-12T18:14:45.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse3 2026-02-12T18:14:45.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:14:45.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:14:45.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:14:45.598 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:45.594+0000 7f10afc2c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:45.599 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:45.595+0000 7f10afc2c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:45.600 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:45.596+0000 7f10afc2c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:45.680 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T18:14:45.680 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:14:45.680 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T18:14:45.680 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:14:45.680 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:14:45.680 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:45.680 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:14:45.680 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:45.681 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:45.681 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:14:45.774 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:14:45.810 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:45.805+0000 7f10afc2c780 -1 Falling back to public interface 2026-02-12T18:14:46.133 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:46.129+0000 7f10afc2c780 -1 osd.0 0 log_to_monitors true 2026-02-12T18:14:46.775 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:14:46.776 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:46.776 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:14:46.777 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:14:46.777 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:46.777 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:14:46.869 INFO:tasks.workunit.client.0.trial116.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3772920772,v1:127.0.0.1:6801/3772920772] [v2:127.0.0.1:6802/3772920772,v1:127.0.0.1:6803/3772920772] exists,up 76d49fc6-66bd-490e-b703-6e6cd79a5fbd 2026-02-12T18:14:46.871 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:14:46.871 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:14:46.871 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:14:46.871 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-12T18:14:46.871 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T18:14:46.871 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T18:14:46.871 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:46.871 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:46.871 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:46.871 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-osd.0.asok 2026-02-12T18:14:46.872 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-12T18:14:46.872 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.19237/ceph-osd.0.asok log flush 2026-02-12T18:14:46.915 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:14:46.918 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-12T18:14:46.920 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:44.672+0000 7fd71b041c00 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-12T18:14:46.920 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse3' td/test-erasure-code-plugins/osd.0.log 2026-02-12T18:14:46.921 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:45.806+0000 7f10afc2c780 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-12T18:14:46.922 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-12T18:14:46.922 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:46.922 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:14:46.922 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-12T18:14:46.922 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:14:46.923 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:14:46.923 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:14:46.923 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:14:46.923 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:14:47.028 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:14:47.029 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:14:47.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:14:47.030 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:14:47.031 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:14:47.031 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:14:47.031 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:14:47.032 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:47.032 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:14:47.033 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:14:47.033 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:47.034 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:14:47.035 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:14:47.035 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-12T18:14:47.037 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:14:47.037 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:47.038 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:47.038 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19237 2026-02-12T18:14:47.039 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:14:47.040 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:14:47.040 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-12T18:14:47.040 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-12T18:14:47.040 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:47.040 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-12T18:14:47.040 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:47.040 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:14:47.040 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-12T18:14:47.040 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:14:47.040 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:14:47.040 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:14:47.041 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:14:47.041 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:14:47.041 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:14:47.041 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:14:47.042 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:14:47.042 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:14:47.043 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:14:47.043 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:14:47.043 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:14:47.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:47.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:14:47.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:14:47.044 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:47.045 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:14:47.046 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:14:47.046 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-12T18:14:47.047 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:14:47.047 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:47.047 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:47.047 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19237 2026-02-12T18:14:47.048 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:14:47.048 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:14:47.048 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-12T18:14:47.049 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T18:14:47.049 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:47.049 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:47.050 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19237 2026-02-12T18:14:47.050 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T18:14:47.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T18:14:47.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T18:14:47.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-12T18:14:47.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec_sse4 2026-02-12T18:14:47.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:47.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T18:14:47.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T18:14:47.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T18:14:47.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-12T18:14:47.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec_sse4 2026-02-12T18:14:47.070 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T18:14:47.070 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:47.070 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:47.070 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:47.070 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:47.070 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:47.071 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:47.071 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec_sse4 2026-02-12T18:14:47.094 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T18:14:47.095 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T18:14:47.095 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:14:47.095 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:14:47.095 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T18:14:47.095 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T18:14:47.095 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:14:47.096 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:47.096 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:47.096 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:47.097 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:47.097 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:47.097 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:47.097 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:14:47.097 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19237/ceph-mon.a.asok config get fsid 2026-02-12T18:14:47.140 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T18:14:47.141 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:14:47.141 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:14:47.141 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T18:14:47.141 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T18:14:47.141 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:14:47.141 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:47.141 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:47.141 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:47.141 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:47.141 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:47.141 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:47.142 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:14:47.142 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19237/ceph-mon.a.asok config get mon_host 2026-02-12T18:14:47.182 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-12T18:14:47.182 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:47.182 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T18:14:47.183 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T18:14:47.183 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T18:14:47.183 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-12T18:14:47.183 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T18:14:47.282 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T18:14:47.282 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:47.282 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:47.282 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:47.282 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:47.282 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:47.283 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:47.283 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:14:47.283 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:14:47.297 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-12T18:14:47.297 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:47.297 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:47.297 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:47.297 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:47.297 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:47.297 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-mon.a.asok 2026-02-12T18:14:47.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-12T18:14:47.298 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.19237/ceph-mon.a.asok log flush 2026-02-12T18:14:47.336 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:14:47.342 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec_sse4 2026-02-12T18:14:47.343 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:47.343 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:14:47.343 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T18:14:47.343 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:14:47.343 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-12T18:14:47.344 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-12T18:14:47.344 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:14:47.344 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:14:47.344 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:14:47.344 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-12T18:14:47.344 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-12T18:14:47.344 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:14:47.344 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:14:47.344 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-12T18:14:47.344 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:14:47.344 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:47.344 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:47.344 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:47.344 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:47.344 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:47.344 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:47.344 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' 2026-02-12T18:14:47.344 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:14:47.344 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:14:47.345 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:14:47.345 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-12T18:14:47.345 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-12T18:14:47.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:14:47.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:14:47.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:14:47.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:14:47.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:14:47.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec_sse4 2026-02-12T18:14:47.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-12T18:14:47.346 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:14:47.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e3c6977d-1320-4de3-9145-b19504a63b17 2026-02-12T18:14:47.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 e3c6977d-1320-4de3-9145-b19504a63b17' 2026-02-12T18:14:47.346 INFO:tasks.workunit.client.0.trial116.stdout:add osd0 e3c6977d-1320-4de3-9145-b19504a63b17 2026-02-12T18:14:47.346 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:14:47.352 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCXGI5pMS7JFBAA/NW/pdoZNksVkexaffLn+Q== 2026-02-12T18:14:47.352 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCXGI5pMS7JFBAA/NW/pdoZNksVkexaffLn+Q=="}' 2026-02-12T18:14:47.352 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e3c6977d-1320-4de3-9145-b19504a63b17 -i td/test-erasure-code-plugins/0/new.json 2026-02-12T18:14:47.453 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:47.459 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-12T18:14:47.460 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse4 --mkfs --key AQCXGI5pMS7JFBAA/NW/pdoZNksVkexaffLn+Q== --osd-uuid e3c6977d-1320-4de3-9145-b19504a63b17 2026-02-12T18:14:47.475 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:47.471+0000 7f9e4fc2c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:47.477 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:47.473+0000 7f9e4fc2c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:47.478 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:47.474+0000 7f9e4fc2c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:47.479 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:47.474+0000 7f9e4fc2c780 -1 bdev(0x40d6c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-12T18:14:47.479 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:47.474+0000 7f9e4fc2c780 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-12T18:14:47.861 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-12T18:14:47.861 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:14:47.862 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T18:14:47.862 INFO:tasks.workunit.client.0.trial116.stdout:adding osd0 key to auth repository 2026-02-12T18:14:47.863 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:14:47.964 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T18:14:47.964 INFO:tasks.workunit.client.0.trial116.stdout:start osd.0 2026-02-12T18:14:47.965 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=4bbcf1de-50d0-4457-9b1b-8f8accb22a65 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.19237/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse4 2026-02-12T18:14:47.965 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:14:47.965 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:14:47.966 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:14:47.978 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:47.974+0000 7f105c01a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:47.979 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:47.975+0000 7f105c01a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:47.980 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:47.976+0000 7f105c01a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:48.060 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T18:14:48.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:14:48.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T18:14:48.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:14:48.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:14:48.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:48.061 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:48.062 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:14:48.062 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:48.062 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:14:48.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:14:48.213 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:48.209+0000 7f105c01a780 -1 Falling back to public interface 2026-02-12T18:14:48.526 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:48.521+0000 7f105c01a780 -1 osd.0 0 log_to_monitors true 2026-02-12T18:14:49.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:14:49.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:49.160 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:14:49.161 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:14:49.161 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:49.161 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:14:49.260 INFO:tasks.workunit.client.0.trial116.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1459086230,v1:127.0.0.1:6801/1459086230] [v2:127.0.0.1:6802/1459086230,v1:127.0.0.1:6803/1459086230] exists,up e3c6977d-1320-4de3-9145-b19504a63b17 2026-02-12T18:14:49.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:14:49.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:14:49.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:14:49.262 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-12T18:14:49.262 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T18:14:49.262 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T18:14:49.262 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:49.262 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:49.262 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:49.262 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19237/ceph-osd.0.asok 2026-02-12T18:14:49.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-12T18:14:49.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.19237/ceph-osd.0.asok log flush 2026-02-12T18:14:49.304 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:14:49.307 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-12T18:14:49.308 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:47.082+0000 7fa5c5639c00 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-12T18:14:49.309 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse4' td/test-erasure-code-plugins/osd.0.log 2026-02-12T18:14:49.310 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:48.209+0000 7f105c01a780 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-12T18:14:49.311 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-12T18:14:49.311 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:49.311 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:14:49.311 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-12T18:14:49.311 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:14:49.311 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:14:49.311 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:14:49.311 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:14:49.311 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:14:49.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:14:49.417 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:14:49.418 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:14:49.418 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:14:49.420 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:14:49.420 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:14:49.420 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:14:49.421 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:49.421 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:14:49.422 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:14:49.422 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:49.423 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:14:49.424 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:14:49.424 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-12T18:14:49.427 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:14:49.427 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:49.427 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:49.427 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19237 2026-02-12T18:14:49.429 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:14:49.429 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:14:49.429 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:54: TEST_preload_warning: return 0 2026-02-12T18:14:49.429 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-12T18:14:49.429 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/test-erasure-code-plugins 0 2026-02-12T18:14:49.429 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-12T18:14:49.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-12T18:14:49.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-12T18:14:49.430 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:14:49.430 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:14:49.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:14:49.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:14:49.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:14:49.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:14:49.431 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:14:49.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:14:49.431 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:14:49.432 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:14:49.432 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:14:49.432 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:14:49.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:49.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:14:49.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:14:49.434 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:49.434 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:14:49.435 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-12T18:14:49.435 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-12T18:14:49.436 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:14:49.436 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:49.436 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19237 2026-02-12T18:14:49.437 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19237 2026-02-12T18:14:49.438 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:14:49.438 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:14:49.438 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-12T18:14:49.439 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-12T18:14:49.439 DEBUG:teuthology.orchestra.run.trial116:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-12T18:14:49.505 INFO:tasks.workunit:Running workunit erasure-code/test-erasure-code.sh... 2026-02-12T18:14:49.506 DEBUG:teuthology.orchestra.run.trial116:workunit test erasure-code/test-erasure-code.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=afefaa40503a4bf024e96f97dd7f852b75f00ad6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh 2026-02-12T18:14:49.567 INFO:tasks.workunit.client.0.trial116.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-12T18:14:49.571 INFO:tasks.workunit.client.0.trial116.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-12T18:14:49.572 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-12T18:14:49.572 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-12T18:14:49.572 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-12T18:14:49.572 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-12T18:14:49.572 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-12T18:14:49.572 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-12T18:14:49.572 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-12T18:14:49.572 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-12T18:14:49.572 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/test-erasure-code 2026-02-12T18:14:49.572 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:22: run: local dir=td/test-erasure-code 2026-02-12T18:14:49.573 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:23: run: shift 2026-02-12T18:14:49.573 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:25: run: export CEPH_MON=127.0.0.1:7101 2026-02-12T18:14:49.573 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:25: run: CEPH_MON=127.0.0.1:7101 2026-02-12T18:14:49.573 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:26: run: export CEPH_ARGS 2026-02-12T18:14:49.573 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:27: run: uuidgen 2026-02-12T18:14:49.575 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:27: run: CEPH_ARGS+='--fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none ' 2026-02-12T18:14:49.575 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:28: run: CEPH_ARGS+='--mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-12T18:14:49.575 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:30: run: setup td/test-erasure-code 2026-02-12T18:14:49.575 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code 2026-02-12T18:14:49.575 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code 2026-02-12T18:14:49.575 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code 2026-02-12T18:14:49.575 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:14:49.575 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code KILL 2026-02-12T18:14:49.575 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:14:49.575 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:14:49.575 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:14:49.576 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:14:49.576 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:14:49.576 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:14:49.576 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:14:49.577 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:14:49.577 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:14:49.577 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:14:49.577 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:14:49.577 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:14:49.578 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:49.578 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:14:49.578 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:14:49.578 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:14:49.578 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:14:49.579 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:14:49.579 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code 2026-02-12T18:14:49.580 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:14:49.580 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:49.580 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.49040 2026-02-12T18:14:49.580 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.49040 2026-02-12T18:14:49.581 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:14:49.581 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:14:49.581 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code 2026-02-12T18:14:49.582 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T18:14:49.582 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:49.582 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.49040 2026-02-12T18:14:49.582 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.49040 2026-02-12T18:14:49.583 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T18:14:49.584 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-12T18:14:49.584 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-12T18:14:49.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T18:14:49.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code 1' TERM HUP INT 2026-02-12T18:14:49.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:31: run: run_mon td/test-erasure-code a 2026-02-12T18:14:49.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code 2026-02-12T18:14:49.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T18:14:49.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T18:14:49.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T18:14:49.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code/a 2026-02-12T18:14:49.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code/a --run-dir=td/test-erasure-code 2026-02-12T18:14:49.602 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T18:14:49.602 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:49.603 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:49.603 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:49.603 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:49.603 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.49040 2026-02-12T18:14:49.603 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:14:49.603 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code/a '--log-file=td/test-erasure-code/$name.log' '--admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code/log --run-dir=td/test-erasure-code '--pid-file=td/test-erasure-code/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T18:14:49.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T18:14:49.629 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T18:14:49.629 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:14:49.629 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:14:49.629 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T18:14:49.629 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T18:14:49.629 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:14:49.629 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:49.629 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:49.629 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:49.629 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:49.629 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.49040 2026-02-12T18:14:49.630 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.49040/ceph-mon.a.asok 2026-02-12T18:14:49.630 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:14:49.630 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.49040/ceph-mon.a.asok config get fsid 2026-02-12T18:14:49.672 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T18:14:49.672 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:14:49.672 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:14:49.672 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T18:14:49.672 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T18:14:49.672 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:14:49.672 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:49.672 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:49.672 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:49.672 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:49.673 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.49040 2026-02-12T18:14:49.673 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.49040/ceph-mon.a.asok 2026-02-12T18:14:49.673 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:14:49.673 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.49040/ceph-mon.a.asok config get mon_host 2026-02-12T18:14:49.714 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:32: run: run_mgr td/test-erasure-code x 2026-02-12T18:14:49.715 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code 2026-02-12T18:14:49.715 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T18:14:49.715 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T18:14:49.715 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T18:14:49.715 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code/x 2026-02-12T18:14:49.715 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T18:14:49.817 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T18:14:49.817 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:49.817 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:49.817 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:49.817 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:49.817 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.49040 2026-02-12T18:14:49.817 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:14:49.818 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:14:49.818 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code/x '--log-file=td/test-erasure-code/$name.log' '--admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' --run-dir=td/test-erasure-code '--pid-file=td/test-erasure-code/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:14:49.834 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:34: run: get_asok_path mon.a 2026-02-12T18:14:49.834 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:14:49.834 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:14:49.834 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:14:49.834 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:49.834 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.49040 2026-02-12T18:14:49.834 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.49040/ceph-mon.a.asok 2026-02-12T18:14:49.834 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:34: run: CEPH_ARGS= 2026-02-12T18:14:49.835 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:34: run: ceph --admin-daemon /tmp/ceph-asok.49040/ceph-mon.a.asok log flush 2026-02-12T18:14:49.871 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:14:49.875 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:35: run: grep 'load: jerasure.*lrc' td/test-erasure-code/mon.a.log 2026-02-12T18:14:49.876 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:49.616+0000 7fa194e5bc00 0 load: jerasure load: lrc 2026-02-12T18:14:49.877 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: seq 0 10 2026-02-12T18:14:49.880 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-12T18:14:49.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 0 2026-02-12T18:14:49.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-12T18:14:49.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:14:49.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T18:14:49.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:14:49.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/0 2026-02-12T18:14:49.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-12T18:14:49.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:14:49.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:14:49.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:14:49.882 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/0' 2026-02-12T18:14:49.882 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/0/journal' 2026-02-12T18:14:49.882 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:14:49.882 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:14:49.882 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-12T18:14:49.882 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:14:49.882 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:49.882 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:49.882 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:49.882 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:49.882 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.49040 2026-02-12T18:14:49.882 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:14:49.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:14:49.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:14:49.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:14:49.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:14:49.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-12T18:14:49.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-12T18:14:49.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:14:49.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:14:49.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:14:49.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:14:49.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:14:49.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:14:49.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/0 2026-02-12T18:14:49.884 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:14:49.884 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=307b9c46-caa7-4bde-b47e-17fb19e3782a 2026-02-12T18:14:49.884 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 307b9c46-caa7-4bde-b47e-17fb19e3782a' 2026-02-12T18:14:49.884 INFO:tasks.workunit.client.0.trial116.stdout:add osd0 307b9c46-caa7-4bde-b47e-17fb19e3782a 2026-02-12T18:14:49.884 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:14:49.890 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCZGI5pUmrRNBAAHZvG9DVENH2oKcUyY5pYjA== 2026-02-12T18:14:49.890 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCZGI5pUmrRNBAAHZvG9DVENH2oKcUyY5pYjA=="}' 2026-02-12T18:14:49.890 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 307b9c46-caa7-4bde-b47e-17fb19e3782a -i td/test-erasure-code/0/new.json 2026-02-12T18:14:49.993 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:50.000 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/0/new.json 2026-02-12T18:14:50.001 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/0 --osd-journal=td/test-erasure-code/0/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCZGI5pUmrRNBAAHZvG9DVENH2oKcUyY5pYjA== --osd-uuid 307b9c46-caa7-4bde-b47e-17fb19e3782a 2026-02-12T18:14:50.017 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:50.013+0000 7f1c21012780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:50.018 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:50.014+0000 7f1c21012780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:50.019 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:50.015+0000 7f1c21012780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:50.020 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:50.015+0000 7f1c21012780 -1 bdev(0x428ac00 td/test-erasure-code/0/block) open stat got: (1) Operation not permitted 2026-02-12T18:14:50.020 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:50.015+0000 7f1c21012780 -1 bluestore(td/test-erasure-code/0) _read_fsid unparsable uuid 2026-02-12T18:14:50.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/0/keyring 2026-02-12T18:14:50.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:14:50.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T18:14:50.455 INFO:tasks.workunit.client.0.trial116.stdout:adding osd0 key to auth repository 2026-02-12T18:14:50.456 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:14:50.559 INFO:tasks.workunit.client.0.trial116.stdout:start osd.0 2026-02-12T18:14:50.560 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T18:14:50.560 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/0 --osd-journal=td/test-erasure-code/0/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:14:50.560 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:14:50.560 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:14:50.561 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:14:50.574 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:50.570+0000 7f7d4e1c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:50.576 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:50.571+0000 7f7d4e1c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:50.577 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:50.572+0000 7f7d4e1c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:50.655 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T18:14:50.655 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:14:50.656 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T18:14:50.656 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:14:50.656 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:14:50.656 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:50.656 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:50.657 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:14:50.657 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:50.657 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:14:50.751 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:14:50.819 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:50.815+0000 7f7d4e1c6780 -1 Falling back to public interface 2026-02-12T18:14:51.136 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:51.132+0000 7f7d4e1c6780 -1 osd.0 0 log_to_monitors true 2026-02-12T18:14:51.753 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:14:51.753 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:51.753 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:14:51.754 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:14:51.754 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:51.754 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:14:51.852 INFO:tasks.workunit.client.0.trial116.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2862576438,v1:127.0.0.1:6801/2862576438] [v2:127.0.0.1:6802/2862576438,v1:127.0.0.1:6803/2862576438] exists,up 307b9c46-caa7-4bde-b47e-17fb19e3782a 2026-02-12T18:14:51.856 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:14:51.856 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:14:51.856 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:14:51.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-12T18:14:51.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 1 2026-02-12T18:14:51.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-12T18:14:51.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:14:51.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-12T18:14:51.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:14:51.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/1 2026-02-12T18:14:51.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-12T18:14:51.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:14:51.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:14:51.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:14:51.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/1' 2026-02-12T18:14:51.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/1/journal' 2026-02-12T18:14:51.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:14:51.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:14:51.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-12T18:14:51.858 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:14:51.858 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:51.858 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:51.858 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:51.858 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:51.858 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.49040 2026-02-12T18:14:51.858 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:14:51.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:14:51.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:14:51.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:14:51.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:14:51.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-12T18:14:51.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-12T18:14:51.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:14:51.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:14:51.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:14:51.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:14:51.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:14:51.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:14:51.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/1 2026-02-12T18:14:51.859 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:14:51.860 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e437c57b-ee42-47ea-86f8-07726d42819e 2026-02-12T18:14:51.860 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 e437c57b-ee42-47ea-86f8-07726d42819e' 2026-02-12T18:14:51.860 INFO:tasks.workunit.client.0.trial116.stdout:add osd1 e437c57b-ee42-47ea-86f8-07726d42819e 2026-02-12T18:14:51.860 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:14:51.865 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCbGI5pS9ZfMxAAIgPdwY7uxVJxoRz52lbWUg== 2026-02-12T18:14:51.866 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCbGI5pS9ZfMxAAIgPdwY7uxVJxoRz52lbWUg=="}' 2026-02-12T18:14:51.866 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e437c57b-ee42-47ea-86f8-07726d42819e -i td/test-erasure-code/1/new.json 2026-02-12T18:14:52.095 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:14:52.103 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/1/new.json 2026-02-12T18:14:52.104 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/1 --osd-journal=td/test-erasure-code/1/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCbGI5pS9ZfMxAAIgPdwY7uxVJxoRz52lbWUg== --osd-uuid e437c57b-ee42-47ea-86f8-07726d42819e 2026-02-12T18:14:52.119 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:52.115+0000 7f3aa9a63780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:52.120 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:52.116+0000 7f3aa9a63780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:52.121 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:52.117+0000 7f3aa9a63780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:52.122 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:52.117+0000 7f3aa9a63780 -1 bdev(0x410ec00 td/test-erasure-code/1/block) open stat got: (1) Operation not permitted 2026-02-12T18:14:52.122 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:52.117+0000 7f3aa9a63780 -1 bluestore(td/test-erasure-code/1) _read_fsid unparsable uuid 2026-02-12T18:14:52.440 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/1/keyring 2026-02-12T18:14:52.440 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:14:52.441 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-12T18:14:52.441 INFO:tasks.workunit.client.0.trial116.stdout:adding osd1 key to auth repository 2026-02-12T18:14:52.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:14:52.761 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-12T18:14:52.762 INFO:tasks.workunit.client.0.trial116.stdout:start osd.1 2026-02-12T18:14:52.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/1 --osd-journal=td/test-erasure-code/1/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:14:52.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:14:52.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:14:52.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:14:52.775 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:52.771+0000 7f569125e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:52.777 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:52.773+0000 7f569125e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:52.777 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:52.773+0000 7f569125e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:52.951 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:52.946+0000 7f569125e780 -1 Falling back to public interface 2026-02-12T18:14:52.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-12T18:14:52.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:14:52.989 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T18:14:52.989 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:14:52.989 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:14:52.989 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:52.989 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:14:52.989 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:52.990 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:52.990 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:14:53.212 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:14:53.310 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:53.306+0000 7f569125e780 -1 osd.1 0 log_to_monitors true 2026-02-12T18:14:54.213 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:14:54.213 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:54.213 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:14:54.214 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:14:54.214 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:54.214 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:14:54.439 INFO:tasks.workunit.client.0.trial116.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2143770571,v1:127.0.0.1:6811/2143770571] [v2:127.0.0.1:6812/2143770571,v1:127.0.0.1:6813/2143770571] exists,up e437c57b-ee42-47ea-86f8-07726d42819e 2026-02-12T18:14:54.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:14:54.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:14:54.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:14:54.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-12T18:14:54.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 2 2026-02-12T18:14:54.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-12T18:14:54.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:14:54.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-12T18:14:54.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:14:54.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/2 2026-02-12T18:14:54.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-12T18:14:54.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:14:54.445 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:14:54.445 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:14:54.445 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/2' 2026-02-12T18:14:54.445 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/2/journal' 2026-02-12T18:14:54.445 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:14:54.445 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:14:54.445 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-12T18:14:54.445 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:14:54.445 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:54.445 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:54.445 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:54.446 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:54.446 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.49040 2026-02-12T18:14:54.446 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:14:54.446 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:14:54.446 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:14:54.446 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:14:54.446 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:14:54.446 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-12T18:14:54.446 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-12T18:14:54.446 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:14:54.446 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:14:54.446 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:14:54.446 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:14:54.447 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:14:54.447 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:14:54.447 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/2 2026-02-12T18:14:54.447 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:14:54.447 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e438f676-6ab4-4997-8a89-669be87d45ea 2026-02-12T18:14:54.447 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 e438f676-6ab4-4997-8a89-669be87d45ea' 2026-02-12T18:14:54.447 INFO:tasks.workunit.client.0.trial116.stdout:add osd2 e438f676-6ab4-4997-8a89-669be87d45ea 2026-02-12T18:14:54.448 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:14:54.452 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCeGI5p4IrEGhAABbYRW3Qxk1zsd6HzX3A36g== 2026-02-12T18:14:54.453 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCeGI5p4IrEGhAABbYRW3Qxk1zsd6HzX3A36g=="}' 2026-02-12T18:14:54.453 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e438f676-6ab4-4997-8a89-669be87d45ea -i td/test-erasure-code/2/new.json 2026-02-12T18:14:54.687 INFO:tasks.workunit.client.0.trial116.stdout:2 2026-02-12T18:14:54.696 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/2/new.json 2026-02-12T18:14:54.697 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/2 --osd-journal=td/test-erasure-code/2/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCeGI5p4IrEGhAABbYRW3Qxk1zsd6HzX3A36g== --osd-uuid e438f676-6ab4-4997-8a89-669be87d45ea 2026-02-12T18:14:54.712 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:54.708+0000 7f5626013780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:54.713 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:54.709+0000 7f5626013780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:54.714 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:54.710+0000 7f5626013780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:54.715 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:54.710+0000 7f5626013780 -1 bdev(0x3304c00 td/test-erasure-code/2/block) open stat got: (1) Operation not permitted 2026-02-12T18:14:54.715 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:54.711+0000 7f5626013780 -1 bluestore(td/test-erasure-code/2) _read_fsid unparsable uuid 2026-02-12T18:14:55.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/2/keyring 2026-02-12T18:14:55.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:14:55.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-12T18:14:55.030 INFO:tasks.workunit.client.0.trial116.stdout:adding osd2 key to auth repository 2026-02-12T18:14:55.030 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:14:55.363 INFO:tasks.workunit.client.0.trial116.stdout:start osd.2 2026-02-12T18:14:55.364 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-12T18:14:55.364 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/2 --osd-journal=td/test-erasure-code/2/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:14:55.364 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:14:55.365 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:14:55.365 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:14:55.378 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:55.374+0000 7ff171859780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:55.379 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:55.375+0000 7ff171859780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:55.380 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:55.376+0000 7ff171859780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:55.515 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:55.511+0000 7ff171859780 -1 Falling back to public interface 2026-02-12T18:14:55.595 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-12T18:14:55.596 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:14:55.596 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T18:14:55.596 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:14:55.596 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:14:55.596 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:55.596 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:14:55.596 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:55.597 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:55.597 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:14:55.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:14:55.836 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:55.832+0000 7ff171859780 -1 osd.2 0 log_to_monitors true 2026-02-12T18:14:56.829 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:14:56.829 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:56.829 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:14:56.830 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:14:56.830 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:56.830 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:14:57.055 INFO:tasks.workunit.client.0.trial116.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2110419939,v1:127.0.0.1:6819/2110419939] [v2:127.0.0.1:6820/2110419939,v1:127.0.0.1:6821/2110419939] exists,up e438f676-6ab4-4997-8a89-669be87d45ea 2026-02-12T18:14:57.058 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 3 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/3 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/3' 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/3/journal' 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.49040 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:14:57.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:14:57.060 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:14:57.060 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:14:57.060 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:14:57.060 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/3 2026-02-12T18:14:57.060 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:14:57.060 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1b705e0f-2c05-4da2-86d4-4a1ee8129232 2026-02-12T18:14:57.060 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 1b705e0f-2c05-4da2-86d4-4a1ee8129232' 2026-02-12T18:14:57.060 INFO:tasks.workunit.client.0.trial116.stdout:add osd3 1b705e0f-2c05-4da2-86d4-4a1ee8129232 2026-02-12T18:14:57.060 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:14:57.068 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQChGI5p4bTaAxAAj3ytHiAJIGHbrDhwC9DNHQ== 2026-02-12T18:14:57.068 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQChGI5p4bTaAxAAj3ytHiAJIGHbrDhwC9DNHQ=="}' 2026-02-12T18:14:57.068 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1b705e0f-2c05-4da2-86d4-4a1ee8129232 -i td/test-erasure-code/3/new.json 2026-02-12T18:14:57.302 INFO:tasks.workunit.client.0.trial116.stdout:3 2026-02-12T18:14:57.311 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/3/new.json 2026-02-12T18:14:57.312 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/3 --osd-journal=td/test-erasure-code/3/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQChGI5p4bTaAxAAj3ytHiAJIGHbrDhwC9DNHQ== --osd-uuid 1b705e0f-2c05-4da2-86d4-4a1ee8129232 2026-02-12T18:14:57.327 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:57.323+0000 7f75d662c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:57.328 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:57.324+0000 7f75d662c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:57.329 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:57.325+0000 7f75d662c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:57.330 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:57.326+0000 7f75d662c780 -1 bdev(0x2cd6c00 td/test-erasure-code/3/block) open stat got: (1) Operation not permitted 2026-02-12T18:14:57.330 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:57.326+0000 7f75d662c780 -1 bluestore(td/test-erasure-code/3) _read_fsid unparsable uuid 2026-02-12T18:14:57.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/3/keyring 2026-02-12T18:14:57.652 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:14:57.652 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-12T18:14:57.653 INFO:tasks.workunit.client.0.trial116.stdout:adding osd3 key to auth repository 2026-02-12T18:14:57.653 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:14:57.973 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-12T18:14:57.974 INFO:tasks.workunit.client.0.trial116.stdout:start osd.3 2026-02-12T18:14:57.974 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/3 --osd-journal=td/test-erasure-code/3/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:14:57.975 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:14:57.975 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:14:57.975 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:14:57.988 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:57.984+0000 7fc0f9816780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:57.989 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:57.985+0000 7fc0f9816780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:57.990 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:57.986+0000 7fc0f9816780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:58.148 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:58.144+0000 7fc0f9816780 -1 Falling back to public interface 2026-02-12T18:14:58.198 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-12T18:14:58.198 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:14:58.198 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-12T18:14:58.198 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:14:58.199 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:14:58.199 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:58.199 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:14:58.199 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:14:58.199 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:58.200 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:14:58.435 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:14:58.467 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:58.463+0000 7fc0f9816780 -1 osd.3 0 log_to_monitors true 2026-02-12T18:14:59.436 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:14:59.437 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:14:59.437 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:14:59.438 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:14:59.438 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:14:59.438 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:14:59.661 INFO:tasks.workunit.client.0.trial116.stdout:osd.3 up in weight 1 up_from 20 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3417651681,v1:127.0.0.1:6827/3417651681] [v2:127.0.0.1:6828/3417651681,v1:127.0.0.1:6829/3417651681] exists,up 1b705e0f-2c05-4da2-86d4-4a1ee8129232 2026-02-12T18:14:59.665 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:14:59.665 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:14:59.665 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:14:59.665 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-12T18:14:59.666 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 4 2026-02-12T18:14:59.666 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-12T18:14:59.666 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:14:59.666 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-12T18:14:59.666 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:14:59.666 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/4 2026-02-12T18:14:59.666 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-12T18:14:59.666 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:14:59.666 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:14:59.666 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:14:59.666 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/4' 2026-02-12T18:14:59.666 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/4/journal' 2026-02-12T18:14:59.667 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:14:59.667 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:14:59.667 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-12T18:14:59.667 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:14:59.667 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:14:59.667 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:14:59.667 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:14:59.667 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:14:59.667 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.49040 2026-02-12T18:14:59.668 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:14:59.668 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:14:59.668 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:14:59.668 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:14:59.668 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:14:59.668 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-12T18:14:59.668 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-12T18:14:59.668 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:14:59.668 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:14:59.668 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:14:59.668 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:14:59.668 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:14:59.668 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:14:59.669 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/4 2026-02-12T18:14:59.669 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:14:59.669 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5c824c5a-9e13-4332-ba2a-e3aedb6d432b 2026-02-12T18:14:59.669 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 5c824c5a-9e13-4332-ba2a-e3aedb6d432b' 2026-02-12T18:14:59.669 INFO:tasks.workunit.client.0.trial116.stdout:add osd4 5c824c5a-9e13-4332-ba2a-e3aedb6d432b 2026-02-12T18:14:59.669 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:14:59.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCjGI5pl6D/JxAASagBivzwOwBOWOH7FqJMCA== 2026-02-12T18:14:59.675 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCjGI5pl6D/JxAASagBivzwOwBOWOH7FqJMCA=="}' 2026-02-12T18:14:59.675 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5c824c5a-9e13-4332-ba2a-e3aedb6d432b -i td/test-erasure-code/4/new.json 2026-02-12T18:14:59.909 INFO:tasks.workunit.client.0.trial116.stdout:4 2026-02-12T18:14:59.917 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/4/new.json 2026-02-12T18:14:59.918 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/4 --osd-journal=td/test-erasure-code/4/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCjGI5pl6D/JxAASagBivzwOwBOWOH7FqJMCA== --osd-uuid 5c824c5a-9e13-4332-ba2a-e3aedb6d432b 2026-02-12T18:14:59.934 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:59.930+0000 7f368f643780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:59.936 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:59.932+0000 7f368f643780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:59.936 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:59.932+0000 7f368f643780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:14:59.937 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:59.933+0000 7f368f643780 -1 bdev(0x329ac00 td/test-erasure-code/4/block) open stat got: (1) Operation not permitted 2026-02-12T18:14:59.937 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:14:59.933+0000 7f368f643780 -1 bluestore(td/test-erasure-code/4) _read_fsid unparsable uuid 2026-02-12T18:15:00.254 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/4/keyring 2026-02-12T18:15:00.254 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:15:00.255 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-12T18:15:00.255 INFO:tasks.workunit.client.0.trial116.stdout:adding osd4 key to auth repository 2026-02-12T18:15:00.256 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:15:00.574 INFO:tasks.workunit.client.0.trial116.stdout:start osd.4 2026-02-12T18:15:00.575 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-12T18:15:00.576 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/4 --osd-journal=td/test-erasure-code/4/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:15:00.576 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:15:00.576 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:15:00.576 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:15:00.588 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:00.584+0000 7f5a07654780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:00.589 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:00.585+0000 7f5a07654780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:00.590 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:00.586+0000 7f5a07654780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:00.724 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:00.720+0000 7f5a07654780 -1 Falling back to public interface 2026-02-12T18:15:00.808 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-12T18:15:00.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:15:00.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-12T18:15:00.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:15:00.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:15:00.809 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:15:00.810 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:15:00.810 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:15:00.810 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:15:00.810 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-12T18:15:01.045 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:15:01.065 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:01.061+0000 7f5a07654780 -1 osd.4 0 log_to_monitors true 2026-02-12T18:15:02.046 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:15:02.046 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:15:02.047 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:15:02.047 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:15:02.048 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:15:02.048 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-12T18:15:02.272 INFO:tasks.workunit.client.0.trial116.stdout:osd.4 up in weight 1 up_from 25 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/1684906988,v1:127.0.0.1:6835/1684906988] [v2:127.0.0.1:6836/1684906988,v1:127.0.0.1:6837/1684906988] exists,up 5c824c5a-9e13-4332-ba2a-e3aedb6d432b 2026-02-12T18:15:02.276 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:15:02.276 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:15:02.276 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:15:02.276 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-12T18:15:02.276 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 5 2026-02-12T18:15:02.276 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-12T18:15:02.276 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:15:02.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-12T18:15:02.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:15:02.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/5 2026-02-12T18:15:02.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-12T18:15:02.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:15:02.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:15:02.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:15:02.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/5' 2026-02-12T18:15:02.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/5/journal' 2026-02-12T18:15:02.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:15:02.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:15:02.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-12T18:15:02.277 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:15:02.277 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:15:02.277 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:15:02.277 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:15:02.277 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:15:02.277 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.49040 2026-02-12T18:15:02.277 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:15:02.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:15:02.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:15:02.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:15:02.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:15:02.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-12T18:15:02.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-12T18:15:02.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:15:02.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:15:02.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:15:02.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:15:02.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:15:02.278 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:15:02.278 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/5 2026-02-12T18:15:02.278 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:15:02.278 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7e7144b5-a7a3-4c19-9585-a5f2127f68c7 2026-02-12T18:15:02.278 INFO:tasks.workunit.client.0.trial116.stdout:add osd5 7e7144b5-a7a3-4c19-9585-a5f2127f68c7 2026-02-12T18:15:02.279 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 7e7144b5-a7a3-4c19-9585-a5f2127f68c7' 2026-02-12T18:15:02.279 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:15:02.286 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCmGI5pUO/REBAAJhSPl1gu/z4/fWmRovMuww== 2026-02-12T18:15:02.286 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCmGI5pUO/REBAAJhSPl1gu/z4/fWmRovMuww=="}' 2026-02-12T18:15:02.286 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7e7144b5-a7a3-4c19-9585-a5f2127f68c7 -i td/test-erasure-code/5/new.json 2026-02-12T18:15:02.518 INFO:tasks.workunit.client.0.trial116.stdout:5 2026-02-12T18:15:02.525 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/5/new.json 2026-02-12T18:15:02.526 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/5 --osd-journal=td/test-erasure-code/5/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCmGI5pUO/REBAAJhSPl1gu/z4/fWmRovMuww== --osd-uuid 7e7144b5-a7a3-4c19-9585-a5f2127f68c7 2026-02-12T18:15:02.542 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:02.539+0000 7f00e9243780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:02.543 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:02.540+0000 7f00e9243780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:02.544 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:02.541+0000 7f00e9243780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:02.545 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:02.541+0000 7f00e9243780 -1 bdev(0x4208c00 td/test-erasure-code/5/block) open stat got: (1) Operation not permitted 2026-02-12T18:15:02.545 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:02.541+0000 7f00e9243780 -1 bluestore(td/test-erasure-code/5) _read_fsid unparsable uuid 2026-02-12T18:15:02.876 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/5/keyring 2026-02-12T18:15:02.876 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:15:02.877 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-12T18:15:02.877 INFO:tasks.workunit.client.0.trial116.stdout:adding osd5 key to auth repository 2026-02-12T18:15:02.878 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:15:03.193 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-12T18:15:03.194 INFO:tasks.workunit.client.0.trial116.stdout:start osd.5 2026-02-12T18:15:03.195 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/5 --osd-journal=td/test-erasure-code/5/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:15:03.195 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:15:03.195 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:15:03.195 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:15:03.207 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:03.204+0000 7fe4f1e0d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:03.209 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:03.206+0000 7fe4f1e0d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:03.209 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:03.206+0000 7fe4f1e0d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:03.333 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:03.329+0000 7fe4f1e0d780 -1 Falling back to public interface 2026-02-12T18:15:03.418 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-12T18:15:03.418 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:15:03.418 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-12T18:15:03.418 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:15:03.418 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:15:03.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:15:03.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:15:03.419 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:15:03.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:15:03.420 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-12T18:15:03.643 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:03.639+0000 7fe4f1e0d780 -1 osd.5 0 log_to_monitors true 2026-02-12T18:15:03.647 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:15:04.648 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:15:04.648 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:15:04.649 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:15:04.649 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:15:04.649 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:15:04.649 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-12T18:15:04.882 INFO:tasks.workunit.client.0.trial116.stdout:osd.5 up in weight 1 up_from 30 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/301339460,v1:127.0.0.1:6843/301339460] [v2:127.0.0.1:6844/301339460,v1:127.0.0.1:6845/301339460] exists,up 7e7144b5-a7a3-4c19-9585-a5f2127f68c7 2026-02-12T18:15:04.885 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:15:04.885 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:15:04.885 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:15:04.885 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-12T18:15:04.886 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 6 2026-02-12T18:15:04.886 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-12T18:15:04.886 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:15:04.886 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-12T18:15:04.886 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:15:04.886 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/6 2026-02-12T18:15:04.886 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-12T18:15:04.886 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:15:04.886 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:15:04.886 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:15:04.886 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/6' 2026-02-12T18:15:04.886 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/6/journal' 2026-02-12T18:15:04.887 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:15:04.887 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:15:04.887 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-12T18:15:04.887 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:15:04.887 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:15:04.887 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:15:04.887 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:15:04.887 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:15:04.887 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.49040 2026-02-12T18:15:04.887 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:15:04.887 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:15:04.887 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:15:04.887 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:15:04.888 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:15:04.888 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-12T18:15:04.888 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-12T18:15:04.888 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:15:04.888 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:15:04.888 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:15:04.888 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:15:04.888 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:15:04.888 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:15:04.888 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/6 2026-02-12T18:15:04.888 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:15:04.888 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3f4f3567-77f9-436f-b671-17a25ffa8762 2026-02-12T18:15:04.889 INFO:tasks.workunit.client.0.trial116.stdout:add osd6 3f4f3567-77f9-436f-b671-17a25ffa8762 2026-02-12T18:15:04.889 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 3f4f3567-77f9-436f-b671-17a25ffa8762' 2026-02-12T18:15:04.889 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:15:04.895 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCoGI5pWQ0jNRAA4coXmJvfZ3I7CPU6RVgR5w== 2026-02-12T18:15:04.895 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCoGI5pWQ0jNRAA4coXmJvfZ3I7CPU6RVgR5w=="}' 2026-02-12T18:15:04.895 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3f4f3567-77f9-436f-b671-17a25ffa8762 -i td/test-erasure-code/6/new.json 2026-02-12T18:15:05.123 INFO:tasks.workunit.client.0.trial116.stdout:6 2026-02-12T18:15:05.131 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/6/new.json 2026-02-12T18:15:05.132 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/6 --osd-journal=td/test-erasure-code/6/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCoGI5pWQ0jNRAA4coXmJvfZ3I7CPU6RVgR5w== --osd-uuid 3f4f3567-77f9-436f-b671-17a25ffa8762 2026-02-12T18:15:05.147 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:05.144+0000 7f6280c62780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:05.149 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:05.146+0000 7f6280c62780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:05.150 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:05.147+0000 7f6280c62780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:05.150 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:05.147+0000 7f6280c62780 -1 bdev(0x2bf8c00 td/test-erasure-code/6/block) open stat got: (1) Operation not permitted 2026-02-12T18:15:05.150 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:05.147+0000 7f6280c62780 -1 bluestore(td/test-erasure-code/6) _read_fsid unparsable uuid 2026-02-12T18:15:05.480 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/6/keyring 2026-02-12T18:15:05.480 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:15:05.481 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-12T18:15:05.481 INFO:tasks.workunit.client.0.trial116.stdout:adding osd6 key to auth repository 2026-02-12T18:15:05.482 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:15:05.805 INFO:tasks.workunit.client.0.trial116.stdout:start osd.6 2026-02-12T18:15:05.806 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-12T18:15:05.806 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/6 --osd-journal=td/test-erasure-code/6/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:15:05.807 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:15:05.807 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:15:05.807 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:15:05.820 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:05.817+0000 7fc591648780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:05.821 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:05.818+0000 7fc591648780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:05.822 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:05.819+0000 7fc591648780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:06.009 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:06.006+0000 7fc591648780 -1 Falling back to public interface 2026-02-12T18:15:06.033 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-12T18:15:06.033 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:15:06.033 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-12T18:15:06.033 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:15:06.033 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:15:06.033 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:15:06.034 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:15:06.034 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:15:06.034 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:15:06.034 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-12T18:15:06.269 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:15:06.344 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:06.341+0000 7fc591648780 -1 osd.6 0 log_to_monitors true 2026-02-12T18:15:07.271 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:15:07.271 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:15:07.271 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:15:07.272 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:15:07.272 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:15:07.272 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-12T18:15:07.495 INFO:tasks.workunit.client.0.trial116.stdout:osd.6 up in weight 1 up_from 35 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/2844559810,v1:127.0.0.1:6851/2844559810] [v2:127.0.0.1:6852/2844559810,v1:127.0.0.1:6853/2844559810] exists,up 3f4f3567-77f9-436f-b671-17a25ffa8762 2026-02-12T18:15:07.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:15:07.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:15:07.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:15:07.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-12T18:15:07.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 7 2026-02-12T18:15:07.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-12T18:15:07.500 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:15:07.500 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=7 2026-02-12T18:15:07.500 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:15:07.500 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/7 2026-02-12T18:15:07.500 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-12T18:15:07.500 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:15:07.500 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:15:07.500 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:15:07.500 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/7' 2026-02-12T18:15:07.500 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/7/journal' 2026-02-12T18:15:07.500 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:15:07.501 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:15:07.501 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-12T18:15:07.501 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:15:07.501 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:15:07.501 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:15:07.501 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:15:07.501 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:15:07.501 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.49040 2026-02-12T18:15:07.501 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:15:07.501 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:15:07.501 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:15:07.501 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:15:07.501 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:15:07.502 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-12T18:15:07.502 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-12T18:15:07.502 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:15:07.502 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:15:07.502 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:15:07.502 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:15:07.502 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:15:07.502 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:15:07.502 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/7 2026-02-12T18:15:07.502 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:15:07.502 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a1732157-2634-43e7-8c22-1e1ebae5e65d 2026-02-12T18:15:07.502 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd7 a1732157-2634-43e7-8c22-1e1ebae5e65d' 2026-02-12T18:15:07.503 INFO:tasks.workunit.client.0.trial116.stdout:add osd7 a1732157-2634-43e7-8c22-1e1ebae5e65d 2026-02-12T18:15:07.503 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:15:07.509 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCrGI5p3JYiHhAALj/gPnKw8Dys3SN/O3CENg== 2026-02-12T18:15:07.509 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCrGI5p3JYiHhAALj/gPnKw8Dys3SN/O3CENg=="}' 2026-02-12T18:15:07.509 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a1732157-2634-43e7-8c22-1e1ebae5e65d -i td/test-erasure-code/7/new.json 2026-02-12T18:15:07.740 INFO:tasks.workunit.client.0.trial116.stdout:7 2026-02-12T18:15:07.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/7/new.json 2026-02-12T18:15:07.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 7 --fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/7 --osd-journal=td/test-erasure-code/7/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCrGI5p3JYiHhAALj/gPnKw8Dys3SN/O3CENg== --osd-uuid a1732157-2634-43e7-8c22-1e1ebae5e65d 2026-02-12T18:15:07.764 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:07.761+0000 7f7cd8fd8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:07.765 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:07.762+0000 7f7cd8fd8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:07.766 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:07.763+0000 7f7cd8fd8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:07.767 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:07.763+0000 7f7cd8fd8780 -1 bdev(0x327cc00 td/test-erasure-code/7/block) open stat got: (1) Operation not permitted 2026-02-12T18:15:07.767 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:07.763+0000 7f7cd8fd8780 -1 bluestore(td/test-erasure-code/7) _read_fsid unparsable uuid 2026-02-12T18:15:08.092 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/7/keyring 2026-02-12T18:15:08.092 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:15:08.093 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd7 key to auth repository 2026-02-12T18:15:08.093 INFO:tasks.workunit.client.0.trial116.stdout:adding osd7 key to auth repository 2026-02-12T18:15:08.093 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/7/keyring auth add osd.7 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:15:08.407 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.7 2026-02-12T18:15:08.407 INFO:tasks.workunit.client.0.trial116.stdout:start osd.7 2026-02-12T18:15:08.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 7 --fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/7 --osd-journal=td/test-erasure-code/7/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:15:08.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:15:08.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:15:08.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:15:08.421 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:08.418+0000 7f79dce50780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:08.422 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:08.419+0000 7f79dce50780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:08.423 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:08.420+0000 7f79dce50780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:08.580 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:08.577+0000 7f79dce50780 -1 Falling back to public interface 2026-02-12T18:15:08.634 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 7 2026-02-12T18:15:08.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:15:08.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=7 2026-02-12T18:15:08.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:15:08.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:15:08.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:15:08.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:15:08.635 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:15:08.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:15:08.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-12T18:15:08.860 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:15:08.950 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:08.946+0000 7f79dce50780 -1 osd.7 0 log_to_monitors true 2026-02-12T18:15:09.861 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:15:09.861 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:15:09.862 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:15:09.862 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:15:09.862 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:15:09.862 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-12T18:15:10.089 INFO:tasks.workunit.client.0.trial116.stdout:osd.7 up in weight 1 up_from 39 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6858/43987240,v1:127.0.0.1:6859/43987240] [v2:127.0.0.1:6860/43987240,v1:127.0.0.1:6861/43987240] exists,up a1732157-2634-43e7-8c22-1e1ebae5e65d 2026-02-12T18:15:10.094 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:15:10.094 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:15:10.094 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:15:10.094 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-12T18:15:10.094 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 8 2026-02-12T18:15:10.094 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-12T18:15:10.094 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:15:10.095 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=8 2026-02-12T18:15:10.095 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:15:10.095 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/8 2026-02-12T18:15:10.095 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-12T18:15:10.095 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:15:10.095 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:15:10.095 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:15:10.095 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/8' 2026-02-12T18:15:10.095 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/8/journal' 2026-02-12T18:15:10.095 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:15:10.095 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:15:10.095 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-12T18:15:10.096 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:15:10.096 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:15:10.096 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:15:10.096 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:15:10.096 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:15:10.096 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.49040 2026-02-12T18:15:10.096 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:15:10.096 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:15:10.096 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:15:10.096 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:15:10.096 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:15:10.096 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-12T18:15:10.096 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-12T18:15:10.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:15:10.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:15:10.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:15:10.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:15:10.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:15:10.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:15:10.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/8 2026-02-12T18:15:10.097 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:15:10.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f62c223a-e6f8-437f-835f-a4f04d61e0fc 2026-02-12T18:15:10.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd8 f62c223a-e6f8-437f-835f-a4f04d61e0fc' 2026-02-12T18:15:10.098 INFO:tasks.workunit.client.0.trial116.stdout:add osd8 f62c223a-e6f8-437f-835f-a4f04d61e0fc 2026-02-12T18:15:10.098 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:15:10.103 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCuGI5pQknyBRAAhk8fpVZdqQ2F+w+6cdNW6A== 2026-02-12T18:15:10.103 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCuGI5pQknyBRAAhk8fpVZdqQ2F+w+6cdNW6A=="}' 2026-02-12T18:15:10.103 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f62c223a-e6f8-437f-835f-a4f04d61e0fc -i td/test-erasure-code/8/new.json 2026-02-12T18:15:10.332 INFO:tasks.workunit.client.0.trial116.stdout:8 2026-02-12T18:15:10.340 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/8/new.json 2026-02-12T18:15:10.341 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 8 --fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/8 --osd-journal=td/test-erasure-code/8/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCuGI5pQknyBRAAhk8fpVZdqQ2F+w+6cdNW6A== --osd-uuid f62c223a-e6f8-437f-835f-a4f04d61e0fc 2026-02-12T18:15:10.356 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:10.353+0000 7f7109ffd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:10.357 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:10.354+0000 7f7109ffd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:10.358 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:10.355+0000 7f7109ffd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:10.359 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:10.355+0000 7f7109ffd780 -1 bdev(0x2e38c00 td/test-erasure-code/8/block) open stat got: (1) Operation not permitted 2026-02-12T18:15:10.359 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:10.356+0000 7f7109ffd780 -1 bluestore(td/test-erasure-code/8) _read_fsid unparsable uuid 2026-02-12T18:15:10.682 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/8/keyring 2026-02-12T18:15:10.682 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:15:10.683 INFO:tasks.workunit.client.0.trial116.stdout:adding osd8 key to auth repository 2026-02-12T18:15:10.683 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd8 key to auth repository 2026-02-12T18:15:10.683 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/8/keyring auth add osd.8 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:15:11.020 INFO:tasks.workunit.client.0.trial116.stdout:start osd.8 2026-02-12T18:15:11.020 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.8 2026-02-12T18:15:11.021 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 8 --fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/8 --osd-journal=td/test-erasure-code/8/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:15:11.021 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:15:11.021 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:15:11.021 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:15:11.034 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:11.031+0000 7f476500d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:11.035 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:11.032+0000 7f476500d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:11.036 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:11.033+0000 7f476500d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:11.221 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:11.218+0000 7f476500d780 -1 Falling back to public interface 2026-02-12T18:15:11.246 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 8 2026-02-12T18:15:11.246 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:15:11.246 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=8 2026-02-12T18:15:11.246 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:15:11.246 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:15:11.247 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:15:11.247 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:15:11.248 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:15:11.248 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:15:11.248 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-12T18:15:11.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:15:11.593 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:11.590+0000 7f476500d780 -1 osd.8 0 log_to_monitors true 2026-02-12T18:15:12.473 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:15:12.473 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:15:12.474 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:15:12.474 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:15:12.474 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:15:12.474 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-12T18:15:12.701 INFO:tasks.workunit.client.0.trial116.stdout:osd.8 up in weight 1 up_from 44 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6866/2682253298,v1:127.0.0.1:6867/2682253298] [v2:127.0.0.1:6868/2682253298,v1:127.0.0.1:6869/2682253298] exists,up f62c223a-e6f8-437f-835f-a4f04d61e0fc 2026-02-12T18:15:12.705 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:15:12.705 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:15:12.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:15:12.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-12T18:15:12.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 9 2026-02-12T18:15:12.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-12T18:15:12.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:15:12.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=9 2026-02-12T18:15:12.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:15:12.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/9 2026-02-12T18:15:12.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-12T18:15:12.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:15:12.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:15:12.707 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:15:12.707 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/9' 2026-02-12T18:15:12.707 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/9/journal' 2026-02-12T18:15:12.707 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:15:12.707 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:15:12.707 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-12T18:15:12.707 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:15:12.707 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:15:12.707 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:15:12.707 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:15:12.707 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:15:12.707 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.49040 2026-02-12T18:15:12.708 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:15:12.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:15:12.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:15:12.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:15:12.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:15:12.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-12T18:15:12.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-12T18:15:12.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:15:12.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:15:12.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:15:12.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:15:12.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:15:12.709 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:15:12.709 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/9 2026-02-12T18:15:12.709 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:15:12.709 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b46d9cd8-f00e-4b42-812a-ef89e437302b 2026-02-12T18:15:12.709 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd9 b46d9cd8-f00e-4b42-812a-ef89e437302b' 2026-02-12T18:15:12.709 INFO:tasks.workunit.client.0.trial116.stdout:add osd9 b46d9cd8-f00e-4b42-812a-ef89e437302b 2026-02-12T18:15:12.710 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:15:12.714 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCwGI5pgRhkKhAAoPd/F/8XUX7duH/MQRSBcQ== 2026-02-12T18:15:12.715 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCwGI5pgRhkKhAAoPd/F/8XUX7duH/MQRSBcQ=="}' 2026-02-12T18:15:12.715 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b46d9cd8-f00e-4b42-812a-ef89e437302b -i td/test-erasure-code/9/new.json 2026-02-12T18:15:12.945 INFO:tasks.workunit.client.0.trial116.stdout:9 2026-02-12T18:15:12.953 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/9/new.json 2026-02-12T18:15:12.954 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 9 --fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/9 --osd-journal=td/test-erasure-code/9/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCwGI5pgRhkKhAAoPd/F/8XUX7duH/MQRSBcQ== --osd-uuid b46d9cd8-f00e-4b42-812a-ef89e437302b 2026-02-12T18:15:12.969 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:12.966+0000 7fe1b862f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:12.971 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:12.968+0000 7fe1b862f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:12.971 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:12.968+0000 7fe1b862f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:12.972 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:12.969+0000 7fe1b862f780 -1 bdev(0x321ac00 td/test-erasure-code/9/block) open stat got: (1) Operation not permitted 2026-02-12T18:15:12.972 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:12.969+0000 7fe1b862f780 -1 bluestore(td/test-erasure-code/9) _read_fsid unparsable uuid 2026-02-12T18:15:13.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/9/keyring 2026-02-12T18:15:13.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:15:13.300 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd9 key to auth repository 2026-02-12T18:15:13.300 INFO:tasks.workunit.client.0.trial116.stdout:adding osd9 key to auth repository 2026-02-12T18:15:13.301 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/9/keyring auth add osd.9 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:15:13.618 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.9 2026-02-12T18:15:13.618 INFO:tasks.workunit.client.0.trial116.stdout:start osd.9 2026-02-12T18:15:13.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 9 --fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/9 --osd-journal=td/test-erasure-code/9/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:15:13.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:15:13.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:15:13.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:15:13.632 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:13.629+0000 7fbfeba2f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:13.633 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:13.630+0000 7fbfeba2f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:13.634 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:13.631+0000 7fbfeba2f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:13.823 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:13.820+0000 7fbfeba2f780 -1 Falling back to public interface 2026-02-12T18:15:13.847 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 9 2026-02-12T18:15:13.847 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:15:13.847 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=9 2026-02-12T18:15:13.847 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:15:13.847 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:15:13.847 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:15:13.848 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:15:13.848 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:15:13.848 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:15:13.848 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-12T18:15:14.069 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:15:14.187 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:14.184+0000 7fbfeba2f780 -1 osd.9 0 log_to_monitors true 2026-02-12T18:15:15.070 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:15:15.070 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:15:15.071 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:15:15.071 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:15:15.071 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:15:15.071 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-12T18:15:15.294 INFO:tasks.workunit.client.0.trial116.stdout:osd.9 up in weight 1 up_from 48 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6874/939340634,v1:127.0.0.1:6875/939340634] [v2:127.0.0.1:6876/939340634,v1:127.0.0.1:6877/939340634] exists,up b46d9cd8-f00e-4b42-812a-ef89e437302b 2026-02-12T18:15:15.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:15:15.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:15:15.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:15:15.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-12T18:15:15.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 10 2026-02-12T18:15:15.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-12T18:15:15.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:15:15.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=10 2026-02-12T18:15:15.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:15:15.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/10 2026-02-12T18:15:15.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-12T18:15:15.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:15:15.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:15:15.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:15:15.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/10' 2026-02-12T18:15:15.298 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/10/journal' 2026-02-12T18:15:15.298 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:15:15.298 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:15:15.298 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-12T18:15:15.298 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:15:15.298 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:15:15.298 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:15:15.298 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:15:15.298 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:15:15.298 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.49040 2026-02-12T18:15:15.298 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:15:15.298 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:15:15.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:15:15.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:15:15.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:15:15.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-12T18:15:15.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-12T18:15:15.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:15:15.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:15:15.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:15:15.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:15:15.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:15:15.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:15:15.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/10 2026-02-12T18:15:15.300 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:15:15.300 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ffa5f97c-9620-4516-b1de-d1467a103e34 2026-02-12T18:15:15.300 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd10 ffa5f97c-9620-4516-b1de-d1467a103e34' 2026-02-12T18:15:15.301 INFO:tasks.workunit.client.0.trial116.stdout:add osd10 ffa5f97c-9620-4516-b1de-d1467a103e34 2026-02-12T18:15:15.301 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:15:15.308 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCzGI5pO6MjEhAAyTvPLjJS2nYsvT2ZdvkThA== 2026-02-12T18:15:15.308 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCzGI5pO6MjEhAAyTvPLjJS2nYsvT2ZdvkThA=="}' 2026-02-12T18:15:15.308 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ffa5f97c-9620-4516-b1de-d1467a103e34 -i td/test-erasure-code/10/new.json 2026-02-12T18:15:15.539 INFO:tasks.workunit.client.0.trial116.stdout:10 2026-02-12T18:15:15.548 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/10/new.json 2026-02-12T18:15:15.548 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 10 --fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/10 --osd-journal=td/test-erasure-code/10/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCzGI5pO6MjEhAAyTvPLjJS2nYsvT2ZdvkThA== --osd-uuid ffa5f97c-9620-4516-b1de-d1467a103e34 2026-02-12T18:15:15.564 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:15.560+0000 7f87e7e3d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:15.565 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:15.561+0000 7f87e7e3d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:15.566 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:15.562+0000 7f87e7e3d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:15.567 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:15.563+0000 7f87e7e3d780 -1 bdev(0x423ac00 td/test-erasure-code/10/block) open stat got: (1) Operation not permitted 2026-02-12T18:15:15.567 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:15.563+0000 7f87e7e3d780 -1 bluestore(td/test-erasure-code/10) _read_fsid unparsable uuid 2026-02-12T18:15:15.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/10/keyring 2026-02-12T18:15:15.905 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:15:15.905 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd10 key to auth repository 2026-02-12T18:15:15.906 INFO:tasks.workunit.client.0.trial116.stdout:adding osd10 key to auth repository 2026-02-12T18:15:15.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/10/keyring auth add osd.10 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:15:16.228 INFO:tasks.workunit.client.0.trial116.stdout:start osd.10 2026-02-12T18:15:16.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.10 2026-02-12T18:15:16.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 10 --fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/10 --osd-journal=td/test-erasure-code/10/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:15:16.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:15:16.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:15:16.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:15:16.242 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:16.238+0000 7fd854843780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:16.243 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:16.239+0000 7fd854843780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:16.244 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:16.240+0000 7fd854843780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:16.453 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 10 2026-02-12T18:15:16.453 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:15:16.453 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=10 2026-02-12T18:15:16.453 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:15:16.454 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:15:16.454 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:15:16.454 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:15:16.454 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:15:16.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:15:16.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.10 up' 2026-02-12T18:15:16.473 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:16.469+0000 7fd854843780 -1 Falling back to public interface 2026-02-12T18:15:16.678 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:15:16.771 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:16.767+0000 7fd854843780 -1 osd.10 0 log_to_monitors true 2026-02-12T18:15:17.680 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:15:17.680 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:15:17.680 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:15:17.681 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:15:17.681 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:15:17.681 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.10 up' 2026-02-12T18:15:17.905 INFO:tasks.workunit.client.0.trial116.stdout:osd.10 up in weight 1 up_from 53 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6882/2153924260,v1:127.0.0.1:6883/2153924260] [v2:127.0.0.1:6884/2153924260,v1:127.0.0.1:6885/2153924260] exists,up ffa5f97c-9620-4516-b1de-d1467a103e34 2026-02-12T18:15:17.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:15:17.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:15:17.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:15:17.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:39: run: create_rbd_pool 2026-02-12T18:15:17.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-12T18:15:18.128 INFO:tasks.workunit.client.0.trial116.stderr:pool 'rbd' does not exist 2026-02-12T18:15:18.136 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-12T18:15:18.136 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T18:15:18.413 INFO:tasks.workunit.client.0.trial116.stderr:pool 'rbd' already exists 2026-02-12T18:15:18.421 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:15:19.423 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-12T18:15:19.730 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:40: run: wait_for_clean 2026-02-12T18:15:19.730 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:15:19.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:15:19.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:15:19.731 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:15:19.732 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:15:19.732 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:15:19.732 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:15:19.732 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:15:19.732 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:15:19.812 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:15:19.813 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:15:19.813 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:15:19.813 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:15:19.813 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:15:19.813 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:15:20.042 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:15:20.043 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:15:20.043 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:15:20.043 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:15:20.043 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:15:20.043 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:15:20.043 INFO:tasks.workunit.client.0.trial116.stderr:6 2026-02-12T18:15:20.043 INFO:tasks.workunit.client.0.trial116.stderr:7 2026-02-12T18:15:20.043 INFO:tasks.workunit.client.0.trial116.stderr:8 2026-02-12T18:15:20.043 INFO:tasks.workunit.client.0.trial116.stderr:9 2026-02-12T18:15:20.043 INFO:tasks.workunit.client.0.trial116.stderr:10' 2026-02-12T18:15:20.043 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:15:20.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:20.044 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:15:20.108 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836488 2026-02-12T18:15:20.108 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836488 2026-02-12T18:15:20.108 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488' 2026-02-12T18:15:20.108 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:20.108 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:15:20.173 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705672 2026-02-12T18:15:20.173 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705672 2026-02-12T18:15:20.173 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-38654705672' 2026-02-12T18:15:20.173 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:20.174 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:15:20.237 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574855 2026-02-12T18:15:20.237 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574855 2026-02-12T18:15:20.237 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-38654705672 2-55834574855' 2026-02-12T18:15:20.237 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:20.238 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:15:20.301 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=85899345928 2026-02-12T18:15:20.302 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 85899345928 2026-02-12T18:15:20.302 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-38654705672 2-55834574855 3-85899345928' 2026-02-12T18:15:20.302 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:20.302 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-12T18:15:20.367 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182406 2026-02-12T18:15:20.367 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182406 2026-02-12T18:15:20.367 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-38654705672 2-55834574855 3-85899345928 4-107374182406' 2026-02-12T18:15:20.367 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:20.368 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-12T18:15:20.434 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=128849018886 2026-02-12T18:15:20.434 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 128849018886 2026-02-12T18:15:20.434 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-38654705672 2-55834574855 3-85899345928 4-107374182406 5-128849018886' 2026-02-12T18:15:20.434 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:20.435 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-12T18:15:20.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855365 2026-02-12T18:15:20.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855365 2026-02-12T18:15:20.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-38654705672 2-55834574855 3-85899345928 4-107374182406 5-128849018886 6-150323855365' 2026-02-12T18:15:20.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:20.499 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-12T18:15:20.563 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724549 2026-02-12T18:15:20.563 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724549 2026-02-12T18:15:20.563 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-38654705672 2-55834574855 3-85899345928 4-107374182406 5-128849018886 6-150323855365 7-167503724549' 2026-02-12T18:15:20.563 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:20.563 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-12T18:15:20.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=188978561028 2026-02-12T18:15:20.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 188978561028 2026-02-12T18:15:20.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-38654705672 2-55834574855 3-85899345928 4-107374182406 5-128849018886 6-150323855365 7-167503724549 8-188978561028' 2026-02-12T18:15:20.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:20.628 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-12T18:15:20.692 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=206158430213 2026-02-12T18:15:20.692 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 206158430213 2026-02-12T18:15:20.693 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-38654705672 2-55834574855 3-85899345928 4-107374182406 5-128849018886 6-150323855365 7-167503724549 8-188978561028 9-206158430213' 2026-02-12T18:15:20.693 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:20.693 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-12T18:15:20.759 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=227633266692 2026-02-12T18:15:20.759 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 227633266692 2026-02-12T18:15:20.760 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-38654705672 2-55834574855 3-85899345928 4-107374182406 5-128849018886 6-150323855365 7-167503724549 8-188978561028 9-206158430213 10-227633266692' 2026-02-12T18:15:20.760 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:20.760 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836488 2026-02-12T18:15:20.760 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:20.760 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:15:20.761 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836488 2026-02-12T18:15:20.761 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:20.762 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836488 2026-02-12T18:15:20.762 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836488' 2026-02-12T18:15:20.762 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.0 seq 21474836488 2026-02-12T18:15:20.763 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:15:20.987 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836486 -lt 21474836488 2026-02-12T18:15:20.987 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:15:21.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:15:21.989 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:15:22.220 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836488 -lt 21474836488 2026-02-12T18:15:22.220 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:22.221 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705672 2026-02-12T18:15:22.221 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:22.221 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:15:22.222 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705672 2026-02-12T18:15:22.222 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:22.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705672 2026-02-12T18:15:22.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705672' 2026-02-12T18:15:22.224 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.1 seq 38654705672 2026-02-12T18:15:22.224 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:15:22.447 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705672 -lt 38654705672 2026-02-12T18:15:22.448 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:22.448 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574855 2026-02-12T18:15:22.448 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:22.449 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:15:22.450 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574855 2026-02-12T18:15:22.450 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:22.451 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574855 2026-02-12T18:15:22.451 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574855' 2026-02-12T18:15:22.451 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.2 seq 55834574855 2026-02-12T18:15:22.452 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:15:22.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574855 -lt 55834574855 2026-02-12T18:15:22.687 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:22.687 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-85899345928 2026-02-12T18:15:22.687 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:22.688 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:15:22.689 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-85899345928 2026-02-12T18:15:22.689 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:22.690 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=85899345928 2026-02-12T18:15:22.690 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 85899345928' 2026-02-12T18:15:22.691 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.3 seq 85899345928 2026-02-12T18:15:22.691 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:15:22.927 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 85899345928 -lt 85899345928 2026-02-12T18:15:22.927 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:22.927 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-107374182406 2026-02-12T18:15:22.928 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:22.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-12T18:15:22.929 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-107374182406 2026-02-12T18:15:22.929 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:22.930 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182406 2026-02-12T18:15:22.930 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 107374182406' 2026-02-12T18:15:22.931 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.4 seq 107374182406 2026-02-12T18:15:22.931 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-12T18:15:23.161 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182406 -lt 107374182406 2026-02-12T18:15:23.162 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:23.162 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-128849018886 2026-02-12T18:15:23.162 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:23.163 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-12T18:15:23.164 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-128849018886 2026-02-12T18:15:23.164 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:23.165 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=128849018886 2026-02-12T18:15:23.165 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 128849018886' 2026-02-12T18:15:23.165 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.5 seq 128849018886 2026-02-12T18:15:23.166 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-12T18:15:23.391 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 128849018886 -lt 128849018886 2026-02-12T18:15:23.391 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:23.392 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-150323855365 2026-02-12T18:15:23.392 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:23.392 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-12T18:15:23.393 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-150323855365 2026-02-12T18:15:23.393 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:23.394 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=150323855365 2026-02-12T18:15:23.395 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 150323855365' 2026-02-12T18:15:23.395 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.6 seq 150323855365 2026-02-12T18:15:23.395 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-12T18:15:23.625 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 150323855365 -lt 150323855365 2026-02-12T18:15:23.626 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:23.626 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-167503724549 2026-02-12T18:15:23.626 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:23.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-12T18:15:23.628 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-167503724549 2026-02-12T18:15:23.628 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:23.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=167503724549 2026-02-12T18:15:23.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 167503724549' 2026-02-12T18:15:23.630 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.7 seq 167503724549 2026-02-12T18:15:23.630 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-12T18:15:23.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 167503724549 -lt 167503724549 2026-02-12T18:15:23.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:23.858 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-188978561028 2026-02-12T18:15:23.858 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:23.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-12T18:15:23.860 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-188978561028 2026-02-12T18:15:23.860 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:23.861 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=188978561028 2026-02-12T18:15:23.861 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 188978561028' 2026-02-12T18:15:23.861 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.8 seq 188978561028 2026-02-12T18:15:23.862 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-12T18:15:24.086 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 188978561029 -lt 188978561028 2026-02-12T18:15:24.086 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:24.087 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-206158430213 2026-02-12T18:15:24.087 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:24.087 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-12T18:15:24.088 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-206158430213 2026-02-12T18:15:24.088 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:24.089 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=206158430213 2026-02-12T18:15:24.090 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.9 seq 206158430213 2026-02-12T18:15:24.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 206158430213' 2026-02-12T18:15:24.090 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-12T18:15:24.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 206158430213 -lt 206158430213 2026-02-12T18:15:24.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:24.315 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-227633266692 2026-02-12T18:15:24.315 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:24.316 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-12T18:15:24.317 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-227633266692 2026-02-12T18:15:24.317 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:24.318 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=227633266692 2026-02-12T18:15:24.318 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 227633266692' 2026-02-12T18:15:24.318 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.10 seq 227633266692 2026-02-12T18:15:24.318 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-12T18:15:24.546 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 227633266693 -lt 227633266692 2026-02-12T18:15:24.546 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:15:24.547 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:15:24.547 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:15:24.861 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-12T18:15:24.861 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:15:24.861 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:15:24.861 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:15:24.862 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:15:24.862 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:15:24.862 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:15:24.862 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:15:25.086 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-12T18:15:25.087 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:15:25.087 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:15:25.087 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:15:25.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-12T18:15:25.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:15:25.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:15:25.409 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:42: run: get_asok_path osd.0 2026-02-12T18:15:25.409 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T18:15:25.409 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T18:15:25.409 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:15:25.409 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:15:25.410 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.49040 2026-02-12T18:15:25.410 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.49040/ceph-osd.0.asok 2026-02-12T18:15:25.410 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:42: run: CEPH_ARGS= 2026-02-12T18:15:25.410 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:42: run: ceph --admin-daemon /tmp/ceph-asok.49040/ceph-osd.0.asok log flush 2026-02-12T18:15:25.454 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:15:25.458 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:43: run: grep 'load: jerasure.*lrc' td/test-erasure-code/osd.0.log 2026-02-12T18:15:25.460 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:14:50.817+0000 7f7d4e1c6780 0 load: jerasure load: lrc 2026-02-12T18:15:25.461 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:44: run: create_erasure_coded_pool ecpool 2026-02-12T18:15:25.461 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:56: create_erasure_coded_pool: local poolname=ecpool 2026-02-12T18:15:25.461 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:58: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd 2026-02-12T18:15:25.747 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:60: create_erasure_coded_pool: create_pool ecpool 12 12 erasure myprofile 2026-02-12T18:15:25.747 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create ecpool 12 12 erasure myprofile 2026-02-12T18:15:26.145 INFO:tasks.workunit.client.0.trial116.stderr:pool 'ecpool' already exists 2026-02-12T18:15:26.154 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:15:27.156 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:62: create_erasure_coded_pool: wait_for_clean 2026-02-12T18:15:27.156 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:15:27.156 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:15:27.156 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:15:27.157 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:15:27.157 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:15:27.157 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:15:27.157 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:15:27.157 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:15:27.158 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:15:27.237 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:15:27.237 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:15:27.238 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:15:27.238 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:15:27.238 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:15:27.238 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:15:27.464 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:15:27.464 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:15:27.464 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:15:27.464 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:15:27.465 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:15:27.465 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:15:27.465 INFO:tasks.workunit.client.0.trial116.stderr:6 2026-02-12T18:15:27.465 INFO:tasks.workunit.client.0.trial116.stderr:7 2026-02-12T18:15:27.465 INFO:tasks.workunit.client.0.trial116.stderr:8 2026-02-12T18:15:27.465 INFO:tasks.workunit.client.0.trial116.stderr:9 2026-02-12T18:15:27.465 INFO:tasks.workunit.client.0.trial116.stderr:10' 2026-02-12T18:15:27.465 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:15:27.465 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:27.465 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:15:27.531 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836492 2026-02-12T18:15:27.531 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836492 2026-02-12T18:15:27.532 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492' 2026-02-12T18:15:27.532 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:27.532 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:15:27.596 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705675 2026-02-12T18:15:27.596 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705675 2026-02-12T18:15:27.597 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-38654705675' 2026-02-12T18:15:27.597 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:27.597 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:15:27.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574859 2026-02-12T18:15:27.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574859 2026-02-12T18:15:27.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-38654705675 2-55834574859' 2026-02-12T18:15:27.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:27.664 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:15:27.726 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=85899345931 2026-02-12T18:15:27.726 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 85899345931 2026-02-12T18:15:27.727 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-38654705675 2-55834574859 3-85899345931' 2026-02-12T18:15:27.727 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:27.727 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-12T18:15:27.791 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182410 2026-02-12T18:15:27.791 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182410 2026-02-12T18:15:27.791 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-38654705675 2-55834574859 3-85899345931 4-107374182410' 2026-02-12T18:15:27.791 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:27.791 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-12T18:15:27.855 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=128849018889 2026-02-12T18:15:27.855 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 128849018889 2026-02-12T18:15:27.855 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-38654705675 2-55834574859 3-85899345931 4-107374182410 5-128849018889' 2026-02-12T18:15:27.855 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:27.855 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-12T18:15:27.920 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855369 2026-02-12T18:15:27.921 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855369 2026-02-12T18:15:27.921 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-38654705675 2-55834574859 3-85899345931 4-107374182410 5-128849018889 6-150323855369' 2026-02-12T18:15:27.921 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:27.921 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-12T18:15:27.989 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724552 2026-02-12T18:15:27.989 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724552 2026-02-12T18:15:27.989 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-38654705675 2-55834574859 3-85899345931 4-107374182410 5-128849018889 6-150323855369 7-167503724552' 2026-02-12T18:15:27.989 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:27.989 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-12T18:15:28.053 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=188978561032 2026-02-12T18:15:28.053 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 188978561032 2026-02-12T18:15:28.053 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-38654705675 2-55834574859 3-85899345931 4-107374182410 5-128849018889 6-150323855369 7-167503724552 8-188978561032' 2026-02-12T18:15:28.053 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:28.054 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-12T18:15:28.118 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=206158430216 2026-02-12T18:15:28.118 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 206158430216 2026-02-12T18:15:28.118 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-38654705675 2-55834574859 3-85899345931 4-107374182410 5-128849018889 6-150323855369 7-167503724552 8-188978561032 9-206158430216' 2026-02-12T18:15:28.118 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:28.118 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-12T18:15:28.181 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=227633266696 2026-02-12T18:15:28.181 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 227633266696 2026-02-12T18:15:28.181 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-38654705675 2-55834574859 3-85899345931 4-107374182410 5-128849018889 6-150323855369 7-167503724552 8-188978561032 9-206158430216 10-227633266696' 2026-02-12T18:15:28.182 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:28.182 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836492 2026-02-12T18:15:28.182 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:28.183 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:15:28.183 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836492 2026-02-12T18:15:28.183 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:28.184 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836492 2026-02-12T18:15:28.185 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836492' 2026-02-12T18:15:28.185 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.0 seq 21474836492 2026-02-12T18:15:28.185 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:15:28.412 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836492 -lt 21474836492 2026-02-12T18:15:28.413 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:28.413 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705675 2026-02-12T18:15:28.413 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:28.414 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:15:28.415 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705675 2026-02-12T18:15:28.415 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:28.416 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705675 2026-02-12T18:15:28.416 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705675' 2026-02-12T18:15:28.417 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.1 seq 38654705675 2026-02-12T18:15:28.417 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:15:28.645 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705675 -lt 38654705675 2026-02-12T18:15:28.645 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:28.646 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574859 2026-02-12T18:15:28.646 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:28.646 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:15:28.647 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574859 2026-02-12T18:15:28.647 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:28.648 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574859 2026-02-12T18:15:28.649 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574859' 2026-02-12T18:15:28.649 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.2 seq 55834574859 2026-02-12T18:15:28.649 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:15:28.877 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574859 -lt 55834574859 2026-02-12T18:15:28.878 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:28.878 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-85899345931 2026-02-12T18:15:28.878 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:28.879 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:15:28.879 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-85899345931 2026-02-12T18:15:28.880 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:28.880 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=85899345931 2026-02-12T18:15:28.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 85899345931' 2026-02-12T18:15:28.881 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.3 seq 85899345931 2026-02-12T18:15:28.881 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:15:29.106 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 85899345931 -lt 85899345931 2026-02-12T18:15:29.107 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:29.107 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-107374182410 2026-02-12T18:15:29.107 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:29.108 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-12T18:15:29.109 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-107374182410 2026-02-12T18:15:29.109 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:29.110 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182410 2026-02-12T18:15:29.110 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 107374182410' 2026-02-12T18:15:29.110 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.4 seq 107374182410 2026-02-12T18:15:29.111 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-12T18:15:29.334 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182410 -lt 107374182410 2026-02-12T18:15:29.334 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:29.335 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-128849018889 2026-02-12T18:15:29.335 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:29.335 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-12T18:15:29.336 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-128849018889 2026-02-12T18:15:29.336 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:29.337 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=128849018889 2026-02-12T18:15:29.337 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 128849018889' 2026-02-12T18:15:29.338 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.5 seq 128849018889 2026-02-12T18:15:29.338 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-12T18:15:29.575 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 128849018887 -lt 128849018889 2026-02-12T18:15:29.576 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:15:30.576 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:15:30.577 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-12T18:15:30.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 128849018889 -lt 128849018889 2026-02-12T18:15:30.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:30.803 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-150323855369 2026-02-12T18:15:30.803 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:30.804 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-12T18:15:30.805 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-150323855369 2026-02-12T18:15:30.805 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:30.806 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=150323855369 2026-02-12T18:15:30.806 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.6 seq 150323855369 2026-02-12T18:15:30.807 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 150323855369' 2026-02-12T18:15:30.807 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-12T18:15:31.035 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 150323855369 -lt 150323855369 2026-02-12T18:15:31.035 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:31.036 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-167503724552 2026-02-12T18:15:31.036 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:31.037 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-12T18:15:31.038 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-167503724552 2026-02-12T18:15:31.038 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:31.039 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=167503724552 2026-02-12T18:15:31.039 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 167503724552' 2026-02-12T18:15:31.039 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.7 seq 167503724552 2026-02-12T18:15:31.039 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-12T18:15:31.268 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 167503724552 -lt 167503724552 2026-02-12T18:15:31.268 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:31.269 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-188978561032 2026-02-12T18:15:31.269 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:31.270 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-12T18:15:31.271 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-188978561032 2026-02-12T18:15:31.271 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:31.272 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=188978561032 2026-02-12T18:15:31.272 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 188978561032' 2026-02-12T18:15:31.272 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.8 seq 188978561032 2026-02-12T18:15:31.273 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-12T18:15:31.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 188978561032 -lt 188978561032 2026-02-12T18:15:31.500 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:31.500 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-206158430216 2026-02-12T18:15:31.500 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:31.501 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-12T18:15:31.502 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-206158430216 2026-02-12T18:15:31.502 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:31.503 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=206158430216 2026-02-12T18:15:31.503 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 206158430216' 2026-02-12T18:15:31.503 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.9 seq 206158430216 2026-02-12T18:15:31.504 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-12T18:15:31.734 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 206158430216 -lt 206158430216 2026-02-12T18:15:31.734 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:31.734 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-227633266696 2026-02-12T18:15:31.734 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:31.735 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-12T18:15:31.736 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-227633266696 2026-02-12T18:15:31.736 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:31.737 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=227633266696 2026-02-12T18:15:31.737 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 227633266696' 2026-02-12T18:15:31.738 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.10 seq 227633266696 2026-02-12T18:15:31.738 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-12T18:15:31.963 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 227633266696 -lt 227633266696 2026-02-12T18:15:31.964 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:15:31.964 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:15:31.965 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:15:32.280 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-12T18:15:32.281 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:15:32.282 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:15:32.282 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:15:32.282 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:15:32.282 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:15:32.282 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:15:32.282 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:15:32.503 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-12T18:15:32.504 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:15:32.504 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:15:32.504 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:15:32.825 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-12T18:15:32.825 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:15:32.825 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:15:32.825 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:46: run: set 2026-02-12T18:15:32.826 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:46: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-12T18:15:32.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:46: run: local 'funcs=TEST_alignment_constraints 2026-02-12T18:15:32.827 INFO:tasks.workunit.client.0.trial116.stderr:TEST_chunk_mapping 2026-02-12T18:15:32.827 INFO:tasks.workunit.client.0.trial116.stderr:TEST_rados_put_get_isa 2026-02-12T18:15:32.827 INFO:tasks.workunit.client.0.trial116.stderr:TEST_rados_put_get_jerasure 2026-02-12T18:15:32.827 INFO:tasks.workunit.client.0.trial116.stderr:TEST_rados_put_get_lrc_advanced 2026-02-12T18:15:32.827 INFO:tasks.workunit.client.0.trial116.stderr:TEST_rados_put_get_lrc_kml 2026-02-12T18:15:32.827 INFO:tasks.workunit.client.0.trial116.stderr:TEST_rados_put_get_shec' 2026-02-12T18:15:32.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-12T18:15:32.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_alignment_constraints td/test-erasure-code 2026-02-12T18:15:32.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:266: TEST_alignment_constraints: local payload=ABC 2026-02-12T18:15:32.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:267: TEST_alignment_constraints: echo ABC 2026-02-12T18:15:32.827 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:273: TEST_alignment_constraints: chunk_size 2026-02-12T18:15:32.828 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-12T18:15:32.837 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: local chunk_size=0 2026-02-12T18:15:32.837 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: chunk_size: '[' 0 -eq 0 ']' 2026-02-12T18:15:32.837 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: ceph-conf --show-config-value osd_pool_default_flag_ec_optimizations 2026-02-12T18:15:32.846 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: local ec_opt=false 2026-02-12T18:15:32.846 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:256: chunk_size: '[' false = true ']' 2026-02-12T18:15:32.846 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:259: chunk_size: chunk_size=4096 2026-02-12T18:15:32.846 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: chunk_size: echo 4096 2026-02-12T18:15:32.846 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:273: TEST_alignment_constraints: local stripe_unit=4096 2026-02-12T18:15:32.847 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:274: TEST_alignment_constraints: ceph osd erasure-code-profile get myprofile 2026-02-12T18:15:32.847 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:274: TEST_alignment_constraints: grep k= 2026-02-12T18:15:33.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:274: TEST_alignment_constraints: eval local k=2 2026-02-12T18:15:33.090 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:274: TEST_alignment_constraints: local k=2 2026-02-12T18:15:33.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:275: TEST_alignment_constraints: local block_size=8191 2026-02-12T18:15:33.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:276: TEST_alignment_constraints: dd if=/dev/zero of=td/test-erasure-code/ORIGINAL bs=8191 count=2 2026-02-12T18:15:33.091 INFO:tasks.workunit.client.0.trial116.stderr:2+0 records in 2026-02-12T18:15:33.091 INFO:tasks.workunit.client.0.trial116.stderr:2+0 records out 2026-02-12T18:15:33.091 INFO:tasks.workunit.client.0.trial116.stderr:16382 bytes (16 kB, 16 KiB) copied, 6.6213e-05 s, 247 MB/s 2026-02-12T18:15:33.091 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:277: TEST_alignment_constraints: rados --block-size=8191 --pool ecpool put UNALIGNED td/test-erasure-code/ORIGINAL 2026-02-12T18:15:33.106 INFO:tasks.workunit.client.0.trial116.stderr:INFO: op_size has been rounded to 8192 2026-02-12T18:15:33.110 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:279: TEST_alignment_constraints: rm td/test-erasure-code/ORIGINAL 2026-02-12T18:15:33.111 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-12T18:15:33.111 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_chunk_mapping td/test-erasure-code 2026-02-12T18:15:33.111 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:313: TEST_chunk_mapping: local dir=td/test-erasure-code 2026-02-12T18:15:33.111 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:320: TEST_chunk_mapping: verify_chunk_mapping td/test-erasure-code ecpool 0 1 2026-02-12T18:15:33.111 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:290: verify_chunk_mapping: local dir=td/test-erasure-code 2026-02-12T18:15:33.111 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:291: verify_chunk_mapping: local poolname=ecpool 2026-02-12T18:15:33.111 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: local first=0 2026-02-12T18:15:33.111 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: local second=1 2026-02-12T18:15:33.112 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: chunk_size 2026-02-12T18:15:33.112 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-12T18:15:33.121 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: local chunk_size=0 2026-02-12T18:15:33.121 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: chunk_size: '[' 0 -eq 0 ']' 2026-02-12T18:15:33.121 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: ceph-conf --show-config-value osd_pool_default_flag_ec_optimizations 2026-02-12T18:15:33.130 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: local ec_opt=false 2026-02-12T18:15:33.130 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:256: chunk_size: '[' false = true ']' 2026-02-12T18:15:33.130 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:259: chunk_size: chunk_size=4096 2026-02-12T18:15:33.130 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: chunk_size: echo 4096 2026-02-12T18:15:33.130 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: printf '%*s' 4096 FIRSTecpool 2026-02-12T18:15:33.130 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: chunk_size 2026-02-12T18:15:33.131 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-12T18:15:33.139 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: local chunk_size=0 2026-02-12T18:15:33.139 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: chunk_size: '[' 0 -eq 0 ']' 2026-02-12T18:15:33.139 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: ceph-conf --show-config-value osd_pool_default_flag_ec_optimizations 2026-02-12T18:15:33.148 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: local ec_opt=false 2026-02-12T18:15:33.148 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:256: chunk_size: '[' false = true ']' 2026-02-12T18:15:33.148 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:259: chunk_size: chunk_size=4096 2026-02-12T18:15:33.148 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: chunk_size: echo 4096 2026-02-12T18:15:33.148 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: printf '%*s' 4096 SECONDecpool 2026-02-12T18:15:33.149 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: local 'payload= FIRSTecpool SECONDecpool' 2026-02-12T18:15:33.150 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:296: verify_chunk_mapping: echo -n ' FIRSTecpool SECONDecpool' 2026-02-12T18:15:33.150 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:298: verify_chunk_mapping: rados --pool ecpool put SOMETHINGecpool td/test-erasure-code/ORIGINAL 2026-02-12T18:15:33.165 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:299: verify_chunk_mapping: rados --pool ecpool get SOMETHINGecpool td/test-erasure-code/COPY 2026-02-12T18:15:33.184 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: get_osds ecpool SOMETHINGecpool 2026-02-12T18:15:33.184 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-12T18:15:33.184 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGecpool 2026-02-12T18:15:33.184 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHINGecpool 2026-02-12T18:15:33.185 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:15:33.411 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:15:33.411 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:15:33.411 INFO:tasks.workunit.client.0.trial116.stderr:9 2026-02-12T18:15:33.411 INFO:tasks.workunit.client.0.trial116.stderr:0' 2026-02-12T18:15:33.411 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 9 0 2026-02-12T18:15:33.411 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: osds=('3' '5' '9' '0') 2026-02-12T18:15:33.411 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: local -a osds 2026-02-12T18:15:33.411 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i = 0 )) 2026-02-12T18:15:33.412 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 4 )) 2026-02-12T18:15:33.412 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.3 flush_journal 2026-02-12T18:15:33.461 INFO:tasks.workunit.client.0.trial116.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-12T18:15:33.463 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-12T18:15:33.463 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 4 )) 2026-02-12T18:15:33.463 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.5 flush_journal 2026-02-12T18:15:33.514 INFO:tasks.workunit.client.0.trial116.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-12T18:15:33.516 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-12T18:15:33.516 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 4 )) 2026-02-12T18:15:33.516 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.9 flush_journal 2026-02-12T18:15:33.566 INFO:tasks.workunit.client.0.trial116.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-12T18:15:33.568 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-12T18:15:33.568 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 4 )) 2026-02-12T18:15:33.568 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.0 flush_journal 2026-02-12T18:15:33.618 INFO:tasks.workunit.client.0.trial116.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-12T18:15:33.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-12T18:15:33.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 4 )) 2026-02-12T18:15:33.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:304: verify_chunk_mapping: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-12T18:15:33.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:305: verify_chunk_mapping: rm td/test-erasure-code/COPY 2026-02-12T18:15:33.622 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: get_osds ecpool SOMETHINGecpool 2026-02-12T18:15:33.622 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-12T18:15:33.622 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGecpool 2026-02-12T18:15:33.622 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHINGecpool 2026-02-12T18:15:33.623 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:15:33.853 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:15:33.854 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:15:33.854 INFO:tasks.workunit.client.0.trial116.stderr:9 2026-02-12T18:15:33.854 INFO:tasks.workunit.client.0.trial116.stderr:0' 2026-02-12T18:15:33.854 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 9 0 2026-02-12T18:15:33.854 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: osds=('3' '5' '9' '0') 2026-02-12T18:15:33.854 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: local -a osds 2026-02-12T18:15:33.854 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:308: verify_chunk_mapping: objectstore_tool td/test-erasure-code 3 SOMETHINGecpool get-bytes 2026-02-12T18:15:33.854 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-12T18:15:33.854 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-12T18:15:33.855 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:308: verify_chunk_mapping: grep --quiet FIRSTecpool 2026-02-12T18:15:33.855 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-12T18:15:33.855 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-12T18:15:33.855 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 3 SOMETHINGecpool get-bytes 2026-02-12T18:15:33.855 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-12T18:15:33.855 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-12T18:15:33.855 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-12T18:15:33.855 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-12T18:15:33.855 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.3 2026-02-12T18:15:33.855 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:15:33.855 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:15:33.855 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:15:33.855 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:15:33.856 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:15:34.161 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:15:34.161 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 3 SOMETHINGecpool get-bytes 2026-02-12T18:15:34.161 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-12T18:15:34.161 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-12T18:15:34.161 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-12T18:15:34.161 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-12T18:15:34.161 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/3 2026-02-12T18:15:34.161 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/3 SOMETHINGecpool get-bytes 2026-02-12T18:15:34.260 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 3 2026-02-12T18:15:34.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-12T18:15:34.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T18:15:34.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-12T18:15:34.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T18:15:34.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/3 2026-02-12T18:15:34.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-12T18:15:34.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:15:34.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:15:34.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:15:34.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/3' 2026-02-12T18:15:34.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/3/journal' 2026-02-12T18:15:34.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T18:15:34.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T18:15:34.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-12T18:15:34.262 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T18:15:34.262 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:15:34.262 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:15:34.262 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:15:34.262 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:15:34.262 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.49040 2026-02-12T18:15:34.262 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:15:34.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:15:34.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:15:34.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-12T18:15:34.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-12T18:15:34.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:15:34.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:15:34.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:15:34.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:15:34.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T18:15:34.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-12T18:15:34.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/3 2026-02-12T18:15:34.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-12T18:15:34.263 INFO:tasks.workunit.client.0.trial116.stderr:start osd.3 2026-02-12T18:15:34.264 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/3 --osd-journal=td/test-erasure-code/3/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:15:34.264 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/3/whoami 2026-02-12T18:15:34.264 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-12T18:15:34.264 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T18:15:34.264 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T18:15:34.264 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T18:15:34.273 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:34.269+0000 7f4877e2e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:34.274 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:34.270+0000 7f4877e2e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:34.275 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:34.271+0000 7f4877e2e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:34.420 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:34.416+0000 7f4877e2e780 -1 Falling back to public interface 2026-02-12T18:15:34.487 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-12T18:15:34.487 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:15:34.487 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-12T18:15:34.487 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:15:34.487 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:15:34.488 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:15:34.488 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:15:34.488 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:15:34.488 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:15:34.488 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:15:34.718 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:15:34.775 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:34.772+0000 7f4877e2e780 -1 osd.3 71 log_to_monitors true 2026-02-12T18:15:35.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:15:35.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:15:35.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:15:35.719 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:15:35.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:15:35.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:15:35.946 INFO:tasks.workunit.client.0.trial116.stderr:osd.3 up in weight 1 up_from 74 up_thru 74 down_at 71 last_clean_interval [20,71) [v2:127.0.0.1:6826/3406222214,v1:127.0.0.1:6827/3406222214] [v2:127.0.0.1:6828/3406222214,v1:127.0.0.1:6829/3406222214] exists,up 1b705e0f-2c05-4da2-86d4-4a1ee8129232 2026-02-12T18:15:35.946 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:15:35.946 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:15:35.946 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:15:35.946 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-12T18:15:35.946 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:15:35.946 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:15:35.946 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:15:35.947 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:15:35.947 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:15:35.947 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:15:35.947 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:15:35.947 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:15:35.947 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:15:36.028 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:15:36.028 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:15:36.028 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:15:36.028 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:15:36.028 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:15:36.028 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:15:36.257 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:15:36.257 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:15:36.257 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:15:36.257 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:15:36.257 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:15:36.257 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:15:36.257 INFO:tasks.workunit.client.0.trial116.stderr:6 2026-02-12T18:15:36.257 INFO:tasks.workunit.client.0.trial116.stderr:7 2026-02-12T18:15:36.257 INFO:tasks.workunit.client.0.trial116.stderr:8 2026-02-12T18:15:36.257 INFO:tasks.workunit.client.0.trial116.stderr:9 2026-02-12T18:15:36.257 INFO:tasks.workunit.client.0.trial116.stderr:10' 2026-02-12T18:15:36.257 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:15:36.257 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:36.257 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:15:36.323 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836495 2026-02-12T18:15:36.323 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836495 2026-02-12T18:15:36.323 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495' 2026-02-12T18:15:36.323 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:36.323 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:15:36.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705679 2026-02-12T18:15:36.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705679 2026-02-12T18:15:36.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-38654705679' 2026-02-12T18:15:36.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:36.387 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:15:36.453 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574862 2026-02-12T18:15:36.453 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574862 2026-02-12T18:15:36.453 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-38654705679 2-55834574862' 2026-02-12T18:15:36.453 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:36.453 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:15:36.516 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=317827579907 2026-02-12T18:15:36.516 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 317827579907 2026-02-12T18:15:36.516 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-38654705679 2-55834574862 3-317827579907' 2026-02-12T18:15:36.516 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:36.516 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-12T18:15:36.578 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182413 2026-02-12T18:15:36.578 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182413 2026-02-12T18:15:36.578 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-38654705679 2-55834574862 3-317827579907 4-107374182413' 2026-02-12T18:15:36.578 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:36.578 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-12T18:15:36.642 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=128849018893 2026-02-12T18:15:36.642 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 128849018893 2026-02-12T18:15:36.642 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-38654705679 2-55834574862 3-317827579907 4-107374182413 5-128849018893' 2026-02-12T18:15:36.642 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:36.642 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-12T18:15:36.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855372 2026-02-12T18:15:36.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855372 2026-02-12T18:15:36.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-38654705679 2-55834574862 3-317827579907 4-107374182413 5-128849018893 6-150323855372' 2026-02-12T18:15:36.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:36.706 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-12T18:15:36.771 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724556 2026-02-12T18:15:36.771 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724556 2026-02-12T18:15:36.772 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-38654705679 2-55834574862 3-317827579907 4-107374182413 5-128849018893 6-150323855372 7-167503724556' 2026-02-12T18:15:36.772 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:36.772 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-12T18:15:36.838 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=188978561035 2026-02-12T18:15:36.839 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 188978561035 2026-02-12T18:15:36.839 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-38654705679 2-55834574862 3-317827579907 4-107374182413 5-128849018893 6-150323855372 7-167503724556 8-188978561035' 2026-02-12T18:15:36.839 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:36.839 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-12T18:15:36.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=206158430220 2026-02-12T18:15:36.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 206158430220 2026-02-12T18:15:36.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-38654705679 2-55834574862 3-317827579907 4-107374182413 5-128849018893 6-150323855372 7-167503724556 8-188978561035 9-206158430220' 2026-02-12T18:15:36.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:36.904 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-12T18:15:36.969 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=227633266699 2026-02-12T18:15:36.969 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 227633266699 2026-02-12T18:15:36.969 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-38654705679 2-55834574862 3-317827579907 4-107374182413 5-128849018893 6-150323855372 7-167503724556 8-188978561035 9-206158430220 10-227633266699' 2026-02-12T18:15:36.969 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:36.969 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836495 2026-02-12T18:15:36.969 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:36.970 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:15:36.970 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836495 2026-02-12T18:15:36.971 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:36.971 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836495 2026-02-12T18:15:36.972 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836495' 2026-02-12T18:15:36.972 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.0 seq 21474836495 2026-02-12T18:15:36.972 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:15:37.198 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836493 -lt 21474836495 2026-02-12T18:15:37.198 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:15:38.200 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:15:38.200 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:15:38.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836496 -lt 21474836495 2026-02-12T18:15:38.427 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:38.427 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705679 2026-02-12T18:15:38.427 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:38.427 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:15:38.428 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705679 2026-02-12T18:15:38.428 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:38.429 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705679 2026-02-12T18:15:38.429 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705679' 2026-02-12T18:15:38.429 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.1 seq 38654705679 2026-02-12T18:15:38.430 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:15:38.655 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705679 -lt 38654705679 2026-02-12T18:15:38.655 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:38.656 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574862 2026-02-12T18:15:38.656 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:38.656 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:15:38.657 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574862 2026-02-12T18:15:38.657 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:38.658 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574862 2026-02-12T18:15:38.658 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574862' 2026-02-12T18:15:38.658 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.2 seq 55834574862 2026-02-12T18:15:38.658 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:15:38.885 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574863 -lt 55834574862 2026-02-12T18:15:38.885 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:38.886 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-317827579907 2026-02-12T18:15:38.886 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:38.887 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:15:38.887 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-317827579907 2026-02-12T18:15:38.887 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:38.888 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=317827579907 2026-02-12T18:15:38.888 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 317827579907' 2026-02-12T18:15:38.889 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.3 seq 317827579907 2026-02-12T18:15:38.889 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:15:39.115 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 317827579907 -lt 317827579907 2026-02-12T18:15:39.115 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:39.116 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-107374182413 2026-02-12T18:15:39.116 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:39.117 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-12T18:15:39.117 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-107374182413 2026-02-12T18:15:39.117 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:39.118 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182413 2026-02-12T18:15:39.118 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 107374182413' 2026-02-12T18:15:39.118 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.4 seq 107374182413 2026-02-12T18:15:39.119 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-12T18:15:39.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182414 -lt 107374182413 2026-02-12T18:15:39.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:39.347 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-128849018893 2026-02-12T18:15:39.347 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:39.347 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-12T18:15:39.348 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-128849018893 2026-02-12T18:15:39.348 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:39.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=128849018893 2026-02-12T18:15:39.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 128849018893' 2026-02-12T18:15:39.349 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.5 seq 128849018893 2026-02-12T18:15:39.349 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-12T18:15:39.574 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 128849018893 -lt 128849018893 2026-02-12T18:15:39.574 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:39.574 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-150323855372 2026-02-12T18:15:39.574 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:39.575 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-12T18:15:39.575 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-150323855372 2026-02-12T18:15:39.575 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:39.576 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=150323855372 2026-02-12T18:15:39.577 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 150323855372' 2026-02-12T18:15:39.577 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.6 seq 150323855372 2026-02-12T18:15:39.577 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-12T18:15:39.808 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 150323855373 -lt 150323855372 2026-02-12T18:15:39.808 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:39.808 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-167503724556 2026-02-12T18:15:39.808 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:39.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-12T18:15:39.810 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-167503724556 2026-02-12T18:15:39.810 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:39.811 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=167503724556 2026-02-12T18:15:39.811 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 167503724556' 2026-02-12T18:15:39.811 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.7 seq 167503724556 2026-02-12T18:15:39.811 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-12T18:15:40.041 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 167503724556 -lt 167503724556 2026-02-12T18:15:40.041 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:40.042 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-188978561035 2026-02-12T18:15:40.042 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:40.042 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-12T18:15:40.043 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-188978561035 2026-02-12T18:15:40.043 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:40.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=188978561035 2026-02-12T18:15:40.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 188978561035' 2026-02-12T18:15:40.044 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.8 seq 188978561035 2026-02-12T18:15:40.044 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-12T18:15:40.274 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 188978561036 -lt 188978561035 2026-02-12T18:15:40.274 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:40.275 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-206158430220 2026-02-12T18:15:40.275 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:40.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-12T18:15:40.276 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-206158430220 2026-02-12T18:15:40.276 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:40.278 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=206158430220 2026-02-12T18:15:40.278 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 206158430220' 2026-02-12T18:15:40.278 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.9 seq 206158430220 2026-02-12T18:15:40.278 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-12T18:15:40.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 206158430220 -lt 206158430220 2026-02-12T18:15:40.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:40.508 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-227633266699 2026-02-12T18:15:40.509 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:40.509 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-12T18:15:40.510 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-227633266699 2026-02-12T18:15:40.510 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:40.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=227633266699 2026-02-12T18:15:40.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 227633266699' 2026-02-12T18:15:40.511 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.10 seq 227633266699 2026-02-12T18:15:40.511 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-12T18:15:40.743 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 227633266700 -lt 227633266699 2026-02-12T18:15:40.743 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:15:40.744 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:15:40.744 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:15:41.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-12T18:15:41.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:15:41.061 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:15:41.061 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:15:41.061 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:15:41.061 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:15:41.061 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:15:41.062 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:15:41.284 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-12T18:15:41.284 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:15:41.285 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:15:41.285 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:15:41.612 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-12T18:15:41.612 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:15:41.612 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:15:41.614 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:309: verify_chunk_mapping: objectstore_tool td/test-erasure-code 5 SOMETHINGecpool get-bytes 2026-02-12T18:15:41.614 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-12T18:15:41.614 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-12T18:15:41.615 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=5 2026-02-12T18:15:41.615 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-12T18:15:41.615 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 5 SOMETHINGecpool get-bytes 2026-02-12T18:15:41.615 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-12T18:15:41.615 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-12T18:15:41.615 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=5 2026-02-12T18:15:41.615 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-12T18:15:41.615 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:309: verify_chunk_mapping: grep --quiet SECONDecpool 2026-02-12T18:15:41.615 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.5 2026-02-12T18:15:41.615 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:15:41.615 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:15:41.615 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:15:41.616 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:15:41.616 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:15:41.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:15:41.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 5 SOMETHINGecpool get-bytes 2026-02-12T18:15:41.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-12T18:15:41.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-12T18:15:41.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=5 2026-02-12T18:15:41.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-12T18:15:41.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/5 2026-02-12T18:15:41.720 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/5 SOMETHINGecpool get-bytes 2026-02-12T18:15:41.824 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 5 2026-02-12T18:15:41.824 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-12T18:15:41.824 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T18:15:41.824 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=5 2026-02-12T18:15:41.825 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T18:15:41.825 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/5 2026-02-12T18:15:41.825 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-12T18:15:41.825 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:15:41.825 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:15:41.825 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:15:41.825 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/5' 2026-02-12T18:15:41.825 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/5/journal' 2026-02-12T18:15:41.825 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T18:15:41.825 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T18:15:41.825 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-12T18:15:41.826 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T18:15:41.826 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:15:41.826 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:15:41.826 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:15:41.826 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:15:41.826 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.49040 2026-02-12T18:15:41.826 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:15:41.826 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:15:41.826 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:15:41.826 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-12T18:15:41.826 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-12T18:15:41.826 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:15:41.826 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:15:41.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:15:41.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:15:41.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T18:15:41.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-12T18:15:41.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/5 2026-02-12T18:15:41.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.5 2026-02-12T18:15:41.827 INFO:tasks.workunit.client.0.trial116.stderr:start osd.5 2026-02-12T18:15:41.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 5 --fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/5 --osd-journal=td/test-erasure-code/5/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:15:41.827 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/5/whoami 2026-02-12T18:15:41.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 5 = 5 ']' 2026-02-12T18:15:41.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T18:15:41.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T18:15:41.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T18:15:41.839 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:41.835+0000 7ff9cb425780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:41.840 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:41.836+0000 7ff9cb425780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:41.841 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:41.837+0000 7ff9cb425780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:41.978 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:41.973+0000 7ff9cb425780 -1 Falling back to public interface 2026-02-12T18:15:42.055 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 5 2026-02-12T18:15:42.055 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:15:42.055 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-12T18:15:42.055 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:15:42.055 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:15:42.055 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:15:42.056 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:15:42.056 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:15:42.056 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:15:42.056 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-12T18:15:42.283 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:15:42.316 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:42.312+0000 7ff9cb425780 -1 osd.5 75 log_to_monitors true 2026-02-12T18:15:43.284 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:15:43.284 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:15:43.284 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:15:43.285 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:15:43.285 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:15:43.285 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-12T18:15:43.512 INFO:tasks.workunit.client.0.trial116.stderr:osd.5 up in weight 1 up_from 79 up_thru 79 down_at 76 last_clean_interval [30,75) [v2:127.0.0.1:6842/3775845138,v1:127.0.0.1:6843/3775845138] [v2:127.0.0.1:6844/3775845138,v1:127.0.0.1:6845/3775845138] exists,up 7e7144b5-a7a3-4c19-9585-a5f2127f68c7 2026-02-12T18:15:43.512 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:15:43.512 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:15:43.513 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:15:43.513 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-12T18:15:43.513 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:15:43.513 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:15:43.513 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:15:43.513 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:15:43.514 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:15:43.514 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:15:43.514 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:15:43.514 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:15:43.514 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:15:43.594 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:15:43.594 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:15:43.594 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:15:43.594 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:15:43.594 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:15:43.595 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:15:43.821 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:15:43.821 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:15:43.821 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:15:43.821 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:15:43.821 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:15:43.821 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:15:43.822 INFO:tasks.workunit.client.0.trial116.stderr:6 2026-02-12T18:15:43.822 INFO:tasks.workunit.client.0.trial116.stderr:7 2026-02-12T18:15:43.822 INFO:tasks.workunit.client.0.trial116.stderr:8 2026-02-12T18:15:43.822 INFO:tasks.workunit.client.0.trial116.stderr:9 2026-02-12T18:15:43.822 INFO:tasks.workunit.client.0.trial116.stderr:10' 2026-02-12T18:15:43.822 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:15:43.822 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:43.822 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:15:43.887 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836499 2026-02-12T18:15:43.887 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836499 2026-02-12T18:15:43.887 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499' 2026-02-12T18:15:43.888 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:43.888 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:15:43.950 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705682 2026-02-12T18:15:43.950 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705682 2026-02-12T18:15:43.950 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499 1-38654705682' 2026-02-12T18:15:43.951 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:43.951 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:15:44.013 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574866 2026-02-12T18:15:44.014 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574866 2026-02-12T18:15:44.014 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499 1-38654705682 2-55834574866' 2026-02-12T18:15:44.014 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:44.014 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:15:44.078 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=317827579910 2026-02-12T18:15:44.078 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 317827579910 2026-02-12T18:15:44.078 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499 1-38654705682 2-55834574866 3-317827579910' 2026-02-12T18:15:44.078 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:44.078 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-12T18:15:44.143 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182417 2026-02-12T18:15:44.144 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182417 2026-02-12T18:15:44.144 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499 1-38654705682 2-55834574866 3-317827579910 4-107374182417' 2026-02-12T18:15:44.144 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:44.144 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-12T18:15:44.207 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=339302416387 2026-02-12T18:15:44.207 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 339302416387 2026-02-12T18:15:44.207 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499 1-38654705682 2-55834574866 3-317827579910 4-107374182417 5-339302416387' 2026-02-12T18:15:44.207 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:44.207 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-12T18:15:44.272 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855376 2026-02-12T18:15:44.272 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855376 2026-02-12T18:15:44.272 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499 1-38654705682 2-55834574866 3-317827579910 4-107374182417 5-339302416387 6-150323855376' 2026-02-12T18:15:44.272 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:44.272 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-12T18:15:44.335 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724559 2026-02-12T18:15:44.335 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724559 2026-02-12T18:15:44.335 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499 1-38654705682 2-55834574866 3-317827579910 4-107374182417 5-339302416387 6-150323855376 7-167503724559' 2026-02-12T18:15:44.335 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:44.335 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-12T18:15:44.399 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=188978561039 2026-02-12T18:15:44.399 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 188978561039 2026-02-12T18:15:44.399 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499 1-38654705682 2-55834574866 3-317827579910 4-107374182417 5-339302416387 6-150323855376 7-167503724559 8-188978561039' 2026-02-12T18:15:44.399 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:44.399 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-12T18:15:44.462 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=206158430223 2026-02-12T18:15:44.462 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 206158430223 2026-02-12T18:15:44.463 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499 1-38654705682 2-55834574866 3-317827579910 4-107374182417 5-339302416387 6-150323855376 7-167503724559 8-188978561039 9-206158430223' 2026-02-12T18:15:44.463 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:44.463 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-12T18:15:44.524 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=227633266703 2026-02-12T18:15:44.524 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 227633266703 2026-02-12T18:15:44.524 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836499 1-38654705682 2-55834574866 3-317827579910 4-107374182417 5-339302416387 6-150323855376 7-167503724559 8-188978561039 9-206158430223 10-227633266703' 2026-02-12T18:15:44.524 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:44.524 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836499 2026-02-12T18:15:44.525 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:44.525 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:15:44.526 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836499 2026-02-12T18:15:44.526 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:44.527 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836499 2026-02-12T18:15:44.527 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836499' 2026-02-12T18:15:44.527 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.0 seq 21474836499 2026-02-12T18:15:44.527 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:15:44.757 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836497 -lt 21474836499 2026-02-12T18:15:44.757 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:15:45.758 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:15:45.759 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:15:45.983 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836499 -lt 21474836499 2026-02-12T18:15:45.983 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:45.983 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705682 2026-02-12T18:15:45.984 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:45.984 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:15:45.985 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705682 2026-02-12T18:15:45.985 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:45.986 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705682 2026-02-12T18:15:45.986 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705682' 2026-02-12T18:15:45.986 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.1 seq 38654705682 2026-02-12T18:15:45.987 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:15:46.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705683 -lt 38654705682 2026-02-12T18:15:46.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:46.216 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574866 2026-02-12T18:15:46.216 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:46.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:15:46.217 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574866 2026-02-12T18:15:46.217 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:46.219 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574866 2026-02-12T18:15:46.219 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574866' 2026-02-12T18:15:46.219 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.2 seq 55834574866 2026-02-12T18:15:46.219 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:15:46.446 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574866 -lt 55834574866 2026-02-12T18:15:46.446 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:46.447 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-317827579910 2026-02-12T18:15:46.447 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:46.447 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:15:46.448 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-317827579910 2026-02-12T18:15:46.448 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:46.449 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=317827579910 2026-02-12T18:15:46.449 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 317827579910' 2026-02-12T18:15:46.449 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.3 seq 317827579910 2026-02-12T18:15:46.449 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:15:46.677 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 317827579911 -lt 317827579910 2026-02-12T18:15:46.678 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:46.678 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-107374182417 2026-02-12T18:15:46.678 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:46.679 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-12T18:15:46.679 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-107374182417 2026-02-12T18:15:46.679 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:46.680 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182417 2026-02-12T18:15:46.681 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 107374182417' 2026-02-12T18:15:46.681 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.4 seq 107374182417 2026-02-12T18:15:46.681 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-12T18:15:46.905 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182417 -lt 107374182417 2026-02-12T18:15:46.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:46.906 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-339302416387 2026-02-12T18:15:46.907 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:46.907 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-12T18:15:46.908 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-339302416387 2026-02-12T18:15:46.908 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:46.909 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=339302416387 2026-02-12T18:15:46.909 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 339302416387' 2026-02-12T18:15:46.909 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.5 seq 339302416387 2026-02-12T18:15:46.909 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-12T18:15:47.132 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 339302416387 -lt 339302416387 2026-02-12T18:15:47.132 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:47.132 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-150323855376 2026-02-12T18:15:47.133 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:47.133 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-12T18:15:47.134 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-150323855376 2026-02-12T18:15:47.134 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:47.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=150323855376 2026-02-12T18:15:47.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 150323855376' 2026-02-12T18:15:47.135 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.6 seq 150323855376 2026-02-12T18:15:47.135 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-12T18:15:47.365 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 150323855376 -lt 150323855376 2026-02-12T18:15:47.365 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:47.366 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-167503724559 2026-02-12T18:15:47.366 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:47.366 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-12T18:15:47.367 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-167503724559 2026-02-12T18:15:47.367 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:47.368 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=167503724559 2026-02-12T18:15:47.368 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 167503724559' 2026-02-12T18:15:47.368 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.7 seq 167503724559 2026-02-12T18:15:47.369 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-12T18:15:47.595 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 167503724560 -lt 167503724559 2026-02-12T18:15:47.595 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:47.596 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-188978561039 2026-02-12T18:15:47.596 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:47.596 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-12T18:15:47.597 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-188978561039 2026-02-12T18:15:47.597 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:47.598 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=188978561039 2026-02-12T18:15:47.598 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 188978561039' 2026-02-12T18:15:47.598 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.8 seq 188978561039 2026-02-12T18:15:47.599 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-12T18:15:47.825 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 188978561039 -lt 188978561039 2026-02-12T18:15:47.826 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:47.826 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-206158430223 2026-02-12T18:15:47.826 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:47.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-12T18:15:47.827 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-206158430223 2026-02-12T18:15:47.828 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:47.829 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=206158430223 2026-02-12T18:15:47.829 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 206158430223' 2026-02-12T18:15:47.829 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.9 seq 206158430223 2026-02-12T18:15:47.829 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-12T18:15:48.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 206158430224 -lt 206158430223 2026-02-12T18:15:48.060 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:48.060 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-227633266703 2026-02-12T18:15:48.060 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:48.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-12T18:15:48.061 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-227633266703 2026-02-12T18:15:48.062 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:48.063 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=227633266703 2026-02-12T18:15:48.063 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 227633266703' 2026-02-12T18:15:48.063 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.10 seq 227633266703 2026-02-12T18:15:48.063 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-12T18:15:48.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 227633266703 -lt 227633266703 2026-02-12T18:15:48.292 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:15:48.292 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:15:48.292 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:15:48.611 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-12T18:15:48.611 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:15:48.611 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:15:48.611 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:15:48.611 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:15:48.611 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:15:48.611 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:15:48.611 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:15:48.833 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-12T18:15:48.833 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:15:48.834 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:15:48.834 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:15:49.150 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-12T18:15:49.151 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:15:49.151 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:15:49.151 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:322: TEST_chunk_mapping: ceph osd erasure-code-profile set remap-profile plugin=lrc 'layers=[ [ "cDD", "" ] ]' mapping=_DD 'crush-steps=[ [ "choose", "osd", 0 ] ]' 2026-02-12T18:15:49.439 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:327: TEST_chunk_mapping: ceph osd erasure-code-profile get remap-profile 2026-02-12T18:15:49.676 INFO:tasks.workunit.client.0.trial116.stdout:crush-device-class= 2026-02-12T18:15:49.676 INFO:tasks.workunit.client.0.trial116.stdout:crush-failure-domain=host 2026-02-12T18:15:49.676 INFO:tasks.workunit.client.0.trial116.stdout:crush-num-failure-domains=0 2026-02-12T18:15:49.676 INFO:tasks.workunit.client.0.trial116.stdout:crush-osds-per-failure-domain=0 2026-02-12T18:15:49.676 INFO:tasks.workunit.client.0.trial116.stdout:crush-root=default 2026-02-12T18:15:49.676 INFO:tasks.workunit.client.0.trial116.stdout:crush-steps=[ [ "choose", "osd", 0 ] ] 2026-02-12T18:15:49.676 INFO:tasks.workunit.client.0.trial116.stdout:k=-1 2026-02-12T18:15:49.677 INFO:tasks.workunit.client.0.trial116.stdout:l=-1 2026-02-12T18:15:49.677 INFO:tasks.workunit.client.0.trial116.stdout:layers=[ [ "cDD", "" ] ] 2026-02-12T18:15:49.677 INFO:tasks.workunit.client.0.trial116.stdout:m=-1 2026-02-12T18:15:49.677 INFO:tasks.workunit.client.0.trial116.stdout:mapping=_DD 2026-02-12T18:15:49.677 INFO:tasks.workunit.client.0.trial116.stdout:plugin=lrc 2026-02-12T18:15:49.683 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:328: TEST_chunk_mapping: create_pool remap-pool 12 12 erasure remap-profile 2026-02-12T18:15:49.683 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create remap-pool 12 12 erasure remap-profile 2026-02-12T18:15:50.024 INFO:tasks.workunit.client.0.trial116.stderr:pool 'remap-pool' already exists 2026-02-12T18:15:50.033 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:15:51.034 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:336: TEST_chunk_mapping: verify_chunk_mapping td/test-erasure-code remap-pool 1 2 2026-02-12T18:15:51.035 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:290: verify_chunk_mapping: local dir=td/test-erasure-code 2026-02-12T18:15:51.035 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:291: verify_chunk_mapping: local poolname=remap-pool 2026-02-12T18:15:51.035 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: local first=1 2026-02-12T18:15:51.035 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: local second=2 2026-02-12T18:15:51.035 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: chunk_size 2026-02-12T18:15:51.035 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-12T18:15:51.044 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: local chunk_size=0 2026-02-12T18:15:51.044 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: chunk_size: '[' 0 -eq 0 ']' 2026-02-12T18:15:51.045 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: ceph-conf --show-config-value osd_pool_default_flag_ec_optimizations 2026-02-12T18:15:51.054 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: local ec_opt=false 2026-02-12T18:15:51.054 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:256: chunk_size: '[' false = true ']' 2026-02-12T18:15:51.054 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:259: chunk_size: chunk_size=4096 2026-02-12T18:15:51.054 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: chunk_size: echo 4096 2026-02-12T18:15:51.054 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: printf '%*s' 4096 FIRSTremap-pool 2026-02-12T18:15:51.054 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: chunk_size 2026-02-12T18:15:51.055 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-12T18:15:51.062 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: local chunk_size=0 2026-02-12T18:15:51.063 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: chunk_size: '[' 0 -eq 0 ']' 2026-02-12T18:15:51.063 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: ceph-conf --show-config-value osd_pool_default_flag_ec_optimizations 2026-02-12T18:15:51.072 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: local ec_opt=false 2026-02-12T18:15:51.072 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:256: chunk_size: '[' false = true ']' 2026-02-12T18:15:51.072 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:259: chunk_size: chunk_size=4096 2026-02-12T18:15:51.072 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: chunk_size: echo 4096 2026-02-12T18:15:51.072 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: printf '%*s' 4096 SECONDremap-pool 2026-02-12T18:15:51.072 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: local 'payload= FIRSTremap-pool SECONDremap-pool' 2026-02-12T18:15:51.073 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:296: verify_chunk_mapping: echo -n ' FIRSTremap-pool SECONDremap-pool' 2026-02-12T18:15:51.073 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:298: verify_chunk_mapping: rados --pool remap-pool put SOMETHINGremap-pool td/test-erasure-code/ORIGINAL 2026-02-12T18:15:51.089 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:299: verify_chunk_mapping: rados --pool remap-pool get SOMETHINGremap-pool td/test-erasure-code/COPY 2026-02-12T18:15:51.105 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: get_osds remap-pool SOMETHINGremap-pool 2026-02-12T18:15:51.105 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=remap-pool 2026-02-12T18:15:51.105 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGremap-pool 2026-02-12T18:15:51.106 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map remap-pool SOMETHINGremap-pool 2026-02-12T18:15:51.106 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:15:51.334 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-12T18:15:51.334 INFO:tasks.workunit.client.0.trial116.stderr:6 2026-02-12T18:15:51.334 INFO:tasks.workunit.client.0.trial116.stderr:9' 2026-02-12T18:15:51.334 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 6 9 2026-02-12T18:15:51.334 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: osds=('4' '6' '9') 2026-02-12T18:15:51.335 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: local -a osds 2026-02-12T18:15:51.335 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i = 0 )) 2026-02-12T18:15:51.335 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 3 )) 2026-02-12T18:15:51.335 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.4 flush_journal 2026-02-12T18:15:51.386 INFO:tasks.workunit.client.0.trial116.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-12T18:15:51.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-12T18:15:51.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 3 )) 2026-02-12T18:15:51.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.6 flush_journal 2026-02-12T18:15:51.438 INFO:tasks.workunit.client.0.trial116.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-12T18:15:51.439 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-12T18:15:51.439 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 3 )) 2026-02-12T18:15:51.439 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.9 flush_journal 2026-02-12T18:15:51.490 INFO:tasks.workunit.client.0.trial116.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-12T18:15:51.491 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-12T18:15:51.492 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 3 )) 2026-02-12T18:15:51.492 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:304: verify_chunk_mapping: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-12T18:15:51.492 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:305: verify_chunk_mapping: rm td/test-erasure-code/COPY 2026-02-12T18:15:51.493 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: get_osds remap-pool SOMETHINGremap-pool 2026-02-12T18:15:51.493 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=remap-pool 2026-02-12T18:15:51.494 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGremap-pool 2026-02-12T18:15:51.494 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map remap-pool SOMETHINGremap-pool 2026-02-12T18:15:51.494 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:15:51.722 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-12T18:15:51.722 INFO:tasks.workunit.client.0.trial116.stderr:6 2026-02-12T18:15:51.723 INFO:tasks.workunit.client.0.trial116.stderr:9' 2026-02-12T18:15:51.723 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 6 9 2026-02-12T18:15:51.723 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: osds=('4' '6' '9') 2026-02-12T18:15:51.723 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: local -a osds 2026-02-12T18:15:51.723 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:308: verify_chunk_mapping: objectstore_tool td/test-erasure-code 6 SOMETHINGremap-pool get-bytes 2026-02-12T18:15:51.723 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:308: verify_chunk_mapping: grep --quiet FIRSTremap-pool 2026-02-12T18:15:51.723 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-12T18:15:51.723 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-12T18:15:51.723 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=6 2026-02-12T18:15:51.723 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-12T18:15:51.723 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 6 SOMETHINGremap-pool get-bytes 2026-02-12T18:15:51.724 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-12T18:15:51.724 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-12T18:15:51.724 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=6 2026-02-12T18:15:51.724 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-12T18:15:51.724 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.6 2026-02-12T18:15:51.724 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:15:51.724 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:15:51.724 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:15:51.724 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:15:51.724 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:15:52.028 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:15:52.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 6 SOMETHINGremap-pool get-bytes 2026-02-12T18:15:52.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-12T18:15:52.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-12T18:15:52.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=6 2026-02-12T18:15:52.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-12T18:15:52.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/6 2026-02-12T18:15:52.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/6 SOMETHINGremap-pool get-bytes 2026-02-12T18:15:52.119 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 6 2026-02-12T18:15:52.119 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-12T18:15:52.119 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T18:15:52.119 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=6 2026-02-12T18:15:52.119 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T18:15:52.119 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/6 2026-02-12T18:15:52.119 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-12T18:15:52.120 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:15:52.120 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:15:52.120 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:15:52.120 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/6' 2026-02-12T18:15:52.120 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/6/journal' 2026-02-12T18:15:52.120 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T18:15:52.120 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T18:15:52.120 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-12T18:15:52.120 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T18:15:52.120 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:15:52.120 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:15:52.120 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:15:52.120 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:15:52.121 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.49040 2026-02-12T18:15:52.121 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:15:52.121 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:15:52.121 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:15:52.121 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-12T18:15:52.121 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-12T18:15:52.121 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:15:52.121 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:15:52.121 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:15:52.121 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:15:52.121 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T18:15:52.121 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-12T18:15:52.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/6 2026-02-12T18:15:52.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.6 2026-02-12T18:15:52.122 INFO:tasks.workunit.client.0.trial116.stderr:start osd.6 2026-02-12T18:15:52.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 6 --fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/6 --osd-journal=td/test-erasure-code/6/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:15:52.122 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/6/whoami 2026-02-12T18:15:52.123 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 6 = 6 ']' 2026-02-12T18:15:52.123 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T18:15:52.123 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T18:15:52.123 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T18:15:52.133 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:52.129+0000 7f66f741f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:52.134 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:52.130+0000 7f66f741f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:52.135 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:52.131+0000 7f66f741f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:52.323 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:52.319+0000 7f66f741f780 -1 Falling back to public interface 2026-02-12T18:15:52.345 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 6 2026-02-12T18:15:52.345 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:15:52.345 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-12T18:15:52.345 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:15:52.345 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:15:52.345 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:15:52.345 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:15:52.345 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:15:52.345 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:15:52.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-12T18:15:52.569 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:15:52.678 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:52.674+0000 7f66f741f780 -1 osd.6 88 log_to_monitors true 2026-02-12T18:15:53.571 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:15:53.571 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:15:53.571 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:15:53.571 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:15:53.571 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:15:53.571 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-12T18:15:53.804 INFO:tasks.workunit.client.0.trial116.stderr:osd.6 up in weight 1 up_from 92 up_thru 92 down_at 89 last_clean_interval [35,88) [v2:127.0.0.1:6850/2143981343,v1:127.0.0.1:6851/2143981343] [v2:127.0.0.1:6852/2143981343,v1:127.0.0.1:6853/2143981343] exists,up 3f4f3567-77f9-436f-b671-17a25ffa8762 2026-02-12T18:15:53.804 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:15:53.804 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:15:53.805 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:15:53.805 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-12T18:15:53.805 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:15:53.805 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:15:53.805 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:15:53.805 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:15:53.805 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:15:53.805 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:15:53.805 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:15:53.805 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:15:53.805 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:15:53.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:15:53.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:15:53.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:15:53.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:15:53.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:15:53.883 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:15:54.121 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:15:54.121 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:15:54.121 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:15:54.121 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:15:54.121 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:15:54.121 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:15:54.121 INFO:tasks.workunit.client.0.trial116.stderr:6 2026-02-12T18:15:54.122 INFO:tasks.workunit.client.0.trial116.stderr:7 2026-02-12T18:15:54.122 INFO:tasks.workunit.client.0.trial116.stderr:8 2026-02-12T18:15:54.122 INFO:tasks.workunit.client.0.trial116.stderr:9 2026-02-12T18:15:54.122 INFO:tasks.workunit.client.0.trial116.stderr:10' 2026-02-12T18:15:54.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:15:54.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:54.122 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:15:54.185 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836503 2026-02-12T18:15:54.185 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836503 2026-02-12T18:15:54.185 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503' 2026-02-12T18:15:54.185 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:54.185 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:15:54.247 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705686 2026-02-12T18:15:54.248 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705686 2026-02-12T18:15:54.248 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503 1-38654705686' 2026-02-12T18:15:54.248 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:54.248 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:15:54.313 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574870 2026-02-12T18:15:54.313 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574870 2026-02-12T18:15:54.313 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-55834574870' 2026-02-12T18:15:54.313 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:54.313 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:15:54.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=317827579914 2026-02-12T18:15:54.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 317827579914 2026-02-12T18:15:54.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-55834574870 3-317827579914' 2026-02-12T18:15:54.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:54.379 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-12T18:15:54.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182421 2026-02-12T18:15:54.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182421 2026-02-12T18:15:54.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-55834574870 3-317827579914 4-107374182421' 2026-02-12T18:15:54.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:54.443 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-12T18:15:54.507 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=339302416391 2026-02-12T18:15:54.507 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 339302416391 2026-02-12T18:15:54.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-55834574870 3-317827579914 4-107374182421 5-339302416391' 2026-02-12T18:15:54.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:54.508 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-12T18:15:54.571 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=395136991235 2026-02-12T18:15:54.571 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 395136991235 2026-02-12T18:15:54.572 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-55834574870 3-317827579914 4-107374182421 5-339302416391 6-395136991235' 2026-02-12T18:15:54.572 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:54.572 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-12T18:15:54.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724563 2026-02-12T18:15:54.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724563 2026-02-12T18:15:54.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-55834574870 3-317827579914 4-107374182421 5-339302416391 6-395136991235 7-167503724563' 2026-02-12T18:15:54.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:54.636 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-12T18:15:54.699 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=188978561043 2026-02-12T18:15:54.699 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 188978561043 2026-02-12T18:15:54.699 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-55834574870 3-317827579914 4-107374182421 5-339302416391 6-395136991235 7-167503724563 8-188978561043' 2026-02-12T18:15:54.699 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:54.699 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-12T18:15:54.762 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=206158430227 2026-02-12T18:15:54.762 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 206158430227 2026-02-12T18:15:54.762 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-55834574870 3-317827579914 4-107374182421 5-339302416391 6-395136991235 7-167503724563 8-188978561043 9-206158430227' 2026-02-12T18:15:54.762 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:15:54.762 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-12T18:15:54.826 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=227633266707 2026-02-12T18:15:54.826 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 227633266707 2026-02-12T18:15:54.826 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-55834574870 3-317827579914 4-107374182421 5-339302416391 6-395136991235 7-167503724563 8-188978561043 9-206158430227 10-227633266707' 2026-02-12T18:15:54.826 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:54.827 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836503 2026-02-12T18:15:54.827 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:54.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:15:54.828 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836503 2026-02-12T18:15:54.829 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:54.829 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836503 2026-02-12T18:15:54.830 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836503' 2026-02-12T18:15:54.830 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.0 seq 21474836503 2026-02-12T18:15:54.830 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:15:55.060 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836501 -lt 21474836503 2026-02-12T18:15:55.060 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:15:56.062 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:15:56.062 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:15:56.288 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836503 -lt 21474836503 2026-02-12T18:15:56.288 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:56.288 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705686 2026-02-12T18:15:56.289 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:56.289 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:15:56.290 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705686 2026-02-12T18:15:56.290 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:56.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705686 2026-02-12T18:15:56.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705686' 2026-02-12T18:15:56.291 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.1 seq 38654705686 2026-02-12T18:15:56.291 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:15:56.514 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705687 -lt 38654705686 2026-02-12T18:15:56.514 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:56.515 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574870 2026-02-12T18:15:56.515 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:56.515 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:15:56.516 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574870 2026-02-12T18:15:56.516 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:56.517 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574870 2026-02-12T18:15:56.517 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574870' 2026-02-12T18:15:56.517 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.2 seq 55834574870 2026-02-12T18:15:56.518 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:15:56.745 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574870 -lt 55834574870 2026-02-12T18:15:56.745 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:56.745 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-317827579914 2026-02-12T18:15:56.745 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:56.746 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:15:56.746 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-317827579914 2026-02-12T18:15:56.747 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:56.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=317827579914 2026-02-12T18:15:56.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 317827579914' 2026-02-12T18:15:56.748 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.3 seq 317827579914 2026-02-12T18:15:56.748 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:15:56.978 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 317827579915 -lt 317827579914 2026-02-12T18:15:56.978 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:56.979 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-107374182421 2026-02-12T18:15:56.979 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:56.979 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-12T18:15:56.980 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-107374182421 2026-02-12T18:15:56.980 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:56.981 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182421 2026-02-12T18:15:56.981 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 107374182421' 2026-02-12T18:15:56.981 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.4 seq 107374182421 2026-02-12T18:15:56.981 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-12T18:15:57.209 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182421 -lt 107374182421 2026-02-12T18:15:57.209 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:57.210 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-339302416391 2026-02-12T18:15:57.210 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:57.210 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-12T18:15:57.211 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-339302416391 2026-02-12T18:15:57.211 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:57.212 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=339302416391 2026-02-12T18:15:57.212 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 339302416391' 2026-02-12T18:15:57.212 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.5 seq 339302416391 2026-02-12T18:15:57.212 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-12T18:15:57.441 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 339302416391 -lt 339302416391 2026-02-12T18:15:57.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:57.442 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-395136991235 2026-02-12T18:15:57.442 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:57.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-12T18:15:57.444 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-395136991235 2026-02-12T18:15:57.444 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:57.445 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=395136991235 2026-02-12T18:15:57.445 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 395136991235' 2026-02-12T18:15:57.445 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.6 seq 395136991235 2026-02-12T18:15:57.445 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-12T18:15:57.672 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 395136991235 -lt 395136991235 2026-02-12T18:15:57.672 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:57.672 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-167503724563 2026-02-12T18:15:57.672 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:57.673 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-12T18:15:57.674 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-167503724563 2026-02-12T18:15:57.674 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:57.675 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=167503724563 2026-02-12T18:15:57.675 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 167503724563' 2026-02-12T18:15:57.675 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.7 seq 167503724563 2026-02-12T18:15:57.675 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-12T18:15:57.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 167503724564 -lt 167503724563 2026-02-12T18:15:57.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:57.906 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-188978561043 2026-02-12T18:15:57.907 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:57.907 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-12T18:15:57.908 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-188978561043 2026-02-12T18:15:57.908 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:57.909 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=188978561043 2026-02-12T18:15:57.909 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 188978561043' 2026-02-12T18:15:57.909 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.8 seq 188978561043 2026-02-12T18:15:57.909 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-12T18:15:58.139 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 188978561044 -lt 188978561043 2026-02-12T18:15:58.139 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:58.140 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-206158430227 2026-02-12T18:15:58.140 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:58.140 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-12T18:15:58.141 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-206158430227 2026-02-12T18:15:58.141 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:58.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=206158430227 2026-02-12T18:15:58.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 206158430227' 2026-02-12T18:15:58.143 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.9 seq 206158430227 2026-02-12T18:15:58.143 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-12T18:15:58.370 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 206158430228 -lt 206158430227 2026-02-12T18:15:58.370 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:15:58.371 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-227633266707 2026-02-12T18:15:58.371 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:15:58.371 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-12T18:15:58.372 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-227633266707 2026-02-12T18:15:58.372 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:15:58.373 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=227633266707 2026-02-12T18:15:58.373 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 227633266707' 2026-02-12T18:15:58.373 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.10 seq 227633266707 2026-02-12T18:15:58.374 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-12T18:15:58.600 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 227633266707 -lt 227633266707 2026-02-12T18:15:58.601 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:15:58.601 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:15:58.601 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:15:58.920 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-12T18:15:58.920 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:15:58.920 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:15:58.920 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:15:58.920 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:15:58.920 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:15:58.920 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:15:58.920 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:15:59.147 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-12T18:15:59.147 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:15:59.148 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:15:59.148 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:15:59.463 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-12T18:15:59.464 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:15:59.464 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:15:59.466 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:309: verify_chunk_mapping: objectstore_tool td/test-erasure-code 9 SOMETHINGremap-pool get-bytes 2026-02-12T18:15:59.466 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-12T18:15:59.466 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-12T18:15:59.466 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=9 2026-02-12T18:15:59.466 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-12T18:15:59.466 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 9 SOMETHINGremap-pool get-bytes 2026-02-12T18:15:59.466 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-12T18:15:59.466 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-12T18:15:59.466 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=9 2026-02-12T18:15:59.467 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:309: verify_chunk_mapping: grep --quiet SECONDremap-pool 2026-02-12T18:15:59.467 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-12T18:15:59.467 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.9 2026-02-12T18:15:59.467 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:15:59.467 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:15:59.467 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:15:59.467 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:15:59.467 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:15:59.570 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:15:59.571 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 9 SOMETHINGremap-pool get-bytes 2026-02-12T18:15:59.571 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-12T18:15:59.571 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-12T18:15:59.571 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=9 2026-02-12T18:15:59.571 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-12T18:15:59.571 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/9 2026-02-12T18:15:59.571 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/9 SOMETHINGremap-pool get-bytes 2026-02-12T18:15:59.672 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 9 2026-02-12T18:15:59.672 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-12T18:15:59.672 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T18:15:59.672 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=9 2026-02-12T18:15:59.672 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T18:15:59.672 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/9 2026-02-12T18:15:59.672 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-12T18:15:59.672 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:15:59.673 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:15:59.673 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:15:59.673 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/9' 2026-02-12T18:15:59.673 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/9/journal' 2026-02-12T18:15:59.673 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T18:15:59.673 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T18:15:59.673 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-12T18:15:59.673 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T18:15:59.673 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:15:59.673 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:15:59.673 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:15:59.673 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:15:59.674 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.49040 2026-02-12T18:15:59.674 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:15:59.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' 2026-02-12T18:15:59.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:15:59.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-12T18:15:59.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-12T18:15:59.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:15:59.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:15:59.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:15:59.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:15:59.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T18:15:59.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-12T18:15:59.675 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/9 2026-02-12T18:15:59.675 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.9 2026-02-12T18:15:59.675 INFO:tasks.workunit.client.0.trial116.stderr:start osd.9 2026-02-12T18:15:59.675 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 9 --fsid=38d6c70e-6b45-4c3c-8270-4d9a7d2d339e --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/9 --osd-journal=td/test-erasure-code/9/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.49040/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:15:59.675 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/9/whoami 2026-02-12T18:15:59.675 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 9 = 9 ']' 2026-02-12T18:15:59.675 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T18:15:59.676 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T18:15:59.676 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T18:15:59.684 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:59.680+0000 7f7f69c2c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:59.685 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:59.681+0000 7f7f69c2c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:59.686 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:59.682+0000 7f7f69c2c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:15:59.849 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:15:59.845+0000 7f7f69c2c780 -1 Falling back to public interface 2026-02-12T18:15:59.901 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 9 2026-02-12T18:15:59.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:15:59.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=9 2026-02-12T18:15:59.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:15:59.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:15:59.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:15:59.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:15:59.902 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:15:59.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:15:59.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-12T18:16:00.125 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:16:00.191 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:16:00.187+0000 7f7f69c2c780 -1 osd.9 94 log_to_monitors true 2026-02-12T18:16:01.127 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:16:01.127 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:16:01.127 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:16:01.127 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:16:01.128 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:16:01.128 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-12T18:16:01.355 INFO:tasks.workunit.client.0.trial116.stderr:osd.9 up in weight 1 up_from 98 up_thru 98 down_at 95 last_clean_interval [48,94) [v2:127.0.0.1:6874/3534618211,v1:127.0.0.1:6875/3534618211] [v2:127.0.0.1:6876/3534618211,v1:127.0.0.1:6877/3534618211] exists,up b46d9cd8-f00e-4b42-812a-ef89e437302b 2026-02-12T18:16:01.355 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:16:01.356 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:16:01.356 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:16:01.356 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-12T18:16:01.356 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:16:01.356 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:16:01.356 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:16:01.356 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:16:01.356 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:16:01.356 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:16:01.356 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:16:01.356 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:16:01.357 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:16:01.435 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:16:01.436 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:16:01.436 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:16:01.436 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:16:01.436 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:16:01.436 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:16:01.659 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:16:01.659 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:16:01.659 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:16:01.659 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:16:01.659 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:16:01.659 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:16:01.659 INFO:tasks.workunit.client.0.trial116.stderr:6 2026-02-12T18:16:01.660 INFO:tasks.workunit.client.0.trial116.stderr:7 2026-02-12T18:16:01.660 INFO:tasks.workunit.client.0.trial116.stderr:8 2026-02-12T18:16:01.660 INFO:tasks.workunit.client.0.trial116.stderr:9 2026-02-12T18:16:01.660 INFO:tasks.workunit.client.0.trial116.stderr:10' 2026-02-12T18:16:01.660 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:16:01.660 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:01.660 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:16:01.723 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836506 2026-02-12T18:16:01.723 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836506 2026-02-12T18:16:01.724 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506' 2026-02-12T18:16:01.724 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:01.724 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:16:01.789 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705690 2026-02-12T18:16:01.789 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705690 2026-02-12T18:16:01.789 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506 1-38654705690' 2026-02-12T18:16:01.789 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:01.789 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:16:01.852 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574873 2026-02-12T18:16:01.852 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574873 2026-02-12T18:16:01.852 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506 1-38654705690 2-55834574873' 2026-02-12T18:16:01.852 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:01.852 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:16:01.917 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=317827579918 2026-02-12T18:16:01.917 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 317827579918 2026-02-12T18:16:01.917 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506 1-38654705690 2-55834574873 3-317827579918' 2026-02-12T18:16:01.917 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:01.917 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-12T18:16:01.980 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182424 2026-02-12T18:16:01.981 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182424 2026-02-12T18:16:01.981 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506 1-38654705690 2-55834574873 3-317827579918 4-107374182424' 2026-02-12T18:16:01.981 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:01.981 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-12T18:16:02.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=339302416394 2026-02-12T18:16:02.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 339302416394 2026-02-12T18:16:02.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506 1-38654705690 2-55834574873 3-317827579918 4-107374182424 5-339302416394' 2026-02-12T18:16:02.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:02.044 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-12T18:16:02.111 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=395136991238 2026-02-12T18:16:02.112 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 395136991238 2026-02-12T18:16:02.112 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506 1-38654705690 2-55834574873 3-317827579918 4-107374182424 5-339302416394 6-395136991238' 2026-02-12T18:16:02.112 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:02.112 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-12T18:16:02.174 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724567 2026-02-12T18:16:02.174 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724567 2026-02-12T18:16:02.174 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506 1-38654705690 2-55834574873 3-317827579918 4-107374182424 5-339302416394 6-395136991238 7-167503724567' 2026-02-12T18:16:02.174 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:02.174 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-12T18:16:02.236 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=188978561046 2026-02-12T18:16:02.236 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 188978561046 2026-02-12T18:16:02.237 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506 1-38654705690 2-55834574873 3-317827579918 4-107374182424 5-339302416394 6-395136991238 7-167503724567 8-188978561046' 2026-02-12T18:16:02.237 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:02.237 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-12T18:16:02.301 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=420906795011 2026-02-12T18:16:02.301 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 420906795011 2026-02-12T18:16:02.302 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506 1-38654705690 2-55834574873 3-317827579918 4-107374182424 5-339302416394 6-395136991238 7-167503724567 8-188978561046 9-420906795011' 2026-02-12T18:16:02.302 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:02.302 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-12T18:16:02.364 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=227633266710 2026-02-12T18:16:02.364 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 227633266710 2026-02-12T18:16:02.364 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836506 1-38654705690 2-55834574873 3-317827579918 4-107374182424 5-339302416394 6-395136991238 7-167503724567 8-188978561046 9-420906795011 10-227633266710' 2026-02-12T18:16:02.365 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:02.365 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836506 2026-02-12T18:16:02.365 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:02.365 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:16:02.366 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836506 2026-02-12T18:16:02.366 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:02.367 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836506 2026-02-12T18:16:02.367 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836506' 2026-02-12T18:16:02.367 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.0 seq 21474836506 2026-02-12T18:16:02.367 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:16:02.593 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836506 -lt 21474836506 2026-02-12T18:16:02.593 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:02.594 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705690 2026-02-12T18:16:02.594 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:02.594 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:16:02.595 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705690 2026-02-12T18:16:02.595 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:02.596 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705690 2026-02-12T18:16:02.596 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705690' 2026-02-12T18:16:02.596 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.1 seq 38654705690 2026-02-12T18:16:02.597 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:16:02.825 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705690 -lt 38654705690 2026-02-12T18:16:02.825 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:02.826 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574873 2026-02-12T18:16:02.826 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:02.826 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:16:02.827 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574873 2026-02-12T18:16:02.827 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:02.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574873 2026-02-12T18:16:02.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574873' 2026-02-12T18:16:02.828 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.2 seq 55834574873 2026-02-12T18:16:02.828 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:16:03.064 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574871 -lt 55834574873 2026-02-12T18:16:03.065 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:16:04.065 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:16:04.066 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:16:04.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574874 -lt 55834574873 2026-02-12T18:16:04.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:04.291 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-317827579918 2026-02-12T18:16:04.291 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:04.292 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:16:04.293 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-317827579918 2026-02-12T18:16:04.293 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:04.294 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=317827579918 2026-02-12T18:16:04.294 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 317827579918' 2026-02-12T18:16:04.294 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.3 seq 317827579918 2026-02-12T18:16:04.294 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:16:04.520 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 317827579918 -lt 317827579918 2026-02-12T18:16:04.520 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:04.520 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-107374182424 2026-02-12T18:16:04.520 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:04.521 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-12T18:16:04.521 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-107374182424 2026-02-12T18:16:04.522 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:04.522 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182424 2026-02-12T18:16:04.523 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 107374182424' 2026-02-12T18:16:04.523 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.4 seq 107374182424 2026-02-12T18:16:04.523 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-12T18:16:04.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182425 -lt 107374182424 2026-02-12T18:16:04.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:04.748 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-339302416394 2026-02-12T18:16:04.748 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:04.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-12T18:16:04.750 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-339302416394 2026-02-12T18:16:04.750 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:04.751 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=339302416394 2026-02-12T18:16:04.751 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 339302416394' 2026-02-12T18:16:04.751 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.5 seq 339302416394 2026-02-12T18:16:04.751 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-12T18:16:04.982 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 339302416395 -lt 339302416394 2026-02-12T18:16:04.982 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:04.982 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-395136991238 2026-02-12T18:16:04.983 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:04.983 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-12T18:16:04.984 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-395136991238 2026-02-12T18:16:04.984 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:04.985 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=395136991238 2026-02-12T18:16:04.985 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 395136991238' 2026-02-12T18:16:04.985 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.6 seq 395136991238 2026-02-12T18:16:04.986 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-12T18:16:05.212 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 395136991239 -lt 395136991238 2026-02-12T18:16:05.212 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:05.213 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-167503724567 2026-02-12T18:16:05.213 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:05.214 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-12T18:16:05.214 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-167503724567 2026-02-12T18:16:05.214 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:05.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=167503724567 2026-02-12T18:16:05.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 167503724567' 2026-02-12T18:16:05.215 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.7 seq 167503724567 2026-02-12T18:16:05.216 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-12T18:16:05.440 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 167503724567 -lt 167503724567 2026-02-12T18:16:05.441 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:05.441 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-188978561046 2026-02-12T18:16:05.441 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:05.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-12T18:16:05.442 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-188978561046 2026-02-12T18:16:05.442 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:05.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=188978561046 2026-02-12T18:16:05.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 188978561046' 2026-02-12T18:16:05.443 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.8 seq 188978561046 2026-02-12T18:16:05.444 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-12T18:16:05.671 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 188978561047 -lt 188978561046 2026-02-12T18:16:05.671 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:05.672 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-420906795011 2026-02-12T18:16:05.672 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:05.673 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-12T18:16:05.673 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-420906795011 2026-02-12T18:16:05.673 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:05.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=420906795011 2026-02-12T18:16:05.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 420906795011' 2026-02-12T18:16:05.675 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.9 seq 420906795011 2026-02-12T18:16:05.675 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-12T18:16:05.908 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 420906795012 -lt 420906795011 2026-02-12T18:16:05.908 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:05.909 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-227633266710 2026-02-12T18:16:05.909 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:05.909 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-12T18:16:05.910 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-227633266710 2026-02-12T18:16:05.910 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:05.911 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=227633266710 2026-02-12T18:16:05.911 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 227633266710' 2026-02-12T18:16:05.911 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.10 seq 227633266710 2026-02-12T18:16:05.911 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-12T18:16:06.139 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 227633266711 -lt 227633266710 2026-02-12T18:16:06.140 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:16:06.140 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:16:06.140 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:16:06.462 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-12T18:16:06.463 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:16:06.463 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:16:06.463 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:16:06.463 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:16:06.463 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:16:06.463 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:16:06.463 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:16:06.687 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-12T18:16:06.688 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:16:06.688 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:16:06.688 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:16:07.022 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-12T18:16:07.022 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:16:07.022 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:16:07.022 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:338: TEST_chunk_mapping: delete_pool remap-pool 2026-02-12T18:16:07.023 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=remap-pool 2026-02-12T18:16:07.023 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete remap-pool remap-pool --yes-i-really-really-mean-it 2026-02-12T18:16:07.260 INFO:tasks.workunit.client.0.trial116.stderr:pool 'remap-pool' does not exist 2026-02-12T18:16:07.268 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:339: TEST_chunk_mapping: ceph osd erasure-code-profile rm remap-profile 2026-02-12T18:16:07.548 INFO:tasks.workunit.client.0.trial116.stderr:erasure-code-profile remap-profile does not exist 2026-02-12T18:16:07.556 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-12T18:16:07.557 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_isa td/test-erasure-code 2026-02-12T18:16:07.557 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:193: TEST_rados_put_get_isa: erasure_code_plugin_exists isa 2026-02-12T18:16:07.557 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2104: erasure_code_plugin_exists: local plugin=isa 2026-02-12T18:16:07.557 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2105: erasure_code_plugin_exists: local status 2026-02-12T18:16:07.557 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2106: erasure_code_plugin_exists: local grepstr 2026-02-12T18:16:07.557 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2107: erasure_code_plugin_exists: local s 2026-02-12T18:16:07.557 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: case `uname` in 2026-02-12T18:16:07.557 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: uname 2026-02-12T18:16:07.558 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2110: erasure_code_plugin_exists: grepstr='isa.*No such file' 2026-02-12T18:16:07.558 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: ceph osd erasure-code-profile set TESTPROFILE plugin=isa 2026-02-12T18:16:07.847 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: s= 2026-02-12T18:16:07.847 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2114: erasure_code_plugin_exists: local status=0 2026-02-12T18:16:07.847 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2115: erasure_code_plugin_exists: '[' 0 -eq 0 ']' 2026-02-12T18:16:07.847 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2116: erasure_code_plugin_exists: ceph osd erasure-code-profile rm TESTPROFILE 2026-02-12T18:16:08.122 INFO:tasks.workunit.client.0.trial116.stderr:erasure-code-profile TESTPROFILE does not exist 2026-02-12T18:16:08.130 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2125: erasure_code_plugin_exists: return 0 2026-02-12T18:16:08.130 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:197: TEST_rados_put_get_isa: local dir=td/test-erasure-code 2026-02-12T18:16:08.130 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:198: TEST_rados_put_get_isa: local poolname=pool-isa 2026-02-12T18:16:08.131 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:200: TEST_rados_put_get_isa: ceph osd erasure-code-profile set profile-isa plugin=isa crush-failure-domain=osd 2026-02-12T18:16:08.422 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:203: TEST_rados_put_get_isa: create_pool pool-isa 1 1 erasure profile-isa 2026-02-12T18:16:08.422 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-isa 1 1 erasure profile-isa 2026-02-12T18:16:08.797 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-isa' already exists 2026-02-12T18:16:08.806 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:16:09.808 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:206: TEST_rados_put_get_isa: rados_put_get td/test-erasure-code pool-isa 2026-02-12T18:16:09.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-12T18:16:09.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-isa 2026-02-12T18:16:09.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-12T18:16:09.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-12T18:16:09.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-12T18:16:09.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-12T18:16:09.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-12T18:16:09.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-12T18:16:09.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-12T18:16:09.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-12T18:16:09.810 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-12T18:16:09.810 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-isa put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-12T18:16:09.876 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-isa get SOMETHING td/test-erasure-code/COPY 2026-02-12T18:16:09.893 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-12T18:16:09.893 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-12T18:16:09.895 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-isa SOMETHING 2026-02-12T18:16:09.895 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-isa 2026-02-12T18:16:09.895 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-12T18:16:09.896 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-isa SOMETHING 2026-02-12T18:16:09.896 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:16:10.124 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-12T18:16:10.124 INFO:tasks.workunit.client.0.trial116.stderr:10 2026-02-12T18:16:10.124 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:16:10.124 INFO:tasks.workunit.client.0.trial116.stderr:7' 2026-02-12T18:16:10.124 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 10 0 7 2026-02-12T18:16:10.124 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('4' '10' '0' '7') 2026-02-12T18:16:10.124 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-12T18:16:10.125 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=3 2026-02-12T18:16:10.125 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 7 2026-02-12T18:16:10.407 INFO:tasks.workunit.client.0.trial116.stderr:osd.7 is already out. 2026-02-12T18:16:10.416 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-12T18:16:15.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-isa SOMETHING 2026-02-12T18:16:15.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-isa 2026-02-12T18:16:15.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-12T18:16:15.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<7\>' 2026-02-12T18:16:15.419 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-isa SOMETHING 2026-02-12T18:16:15.419 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:16:15.654 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-12T18:16:15.654 INFO:tasks.workunit.client.0.trial116.stderr:10 2026-02-12T18:16:15.654 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:16:15.654 INFO:tasks.workunit.client.0.trial116.stderr:1' 2026-02-12T18:16:15.654 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 10 0 1 2026-02-12T18:16:15.654 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-isa get SOMETHING td/test-erasure-code/COPY 2026-02-12T18:16:15.671 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-12T18:16:15.671 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 7 2026-02-12T18:16:15.901 INFO:tasks.workunit.client.0.trial116.stderr:osd.7 is already in. 2026-02-12T18:16:15.909 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-12T18:16:15.910 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:208: TEST_rados_put_get_isa: delete_pool pool-isa 2026-02-12T18:16:15.910 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-isa 2026-02-12T18:16:15.910 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-isa pool-isa --yes-i-really-really-mean-it 2026-02-12T18:16:16.207 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-isa' does not exist 2026-02-12T18:16:16.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-12T18:16:16.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_jerasure td/test-erasure-code 2026-02-12T18:16:16.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:212: TEST_rados_put_get_jerasure: local dir=td/test-erasure-code 2026-02-12T18:16:16.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:214: TEST_rados_put_get_jerasure: rados_put_get td/test-erasure-code ecpool 2026-02-12T18:16:16.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-12T18:16:16.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=ecpool 2026-02-12T18:16:16.218 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-12T18:16:16.218 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-12T18:16:16.218 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-12T18:16:16.218 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-12T18:16:16.218 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-12T18:16:16.218 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-12T18:16:16.218 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-12T18:16:16.218 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-12T18:16:16.218 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-12T18:16:16.218 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool ecpool put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-12T18:16:16.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool ecpool get SOMETHING td/test-erasure-code/COPY 2026-02-12T18:16:16.251 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-12T18:16:16.252 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-12T18:16:16.254 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds ecpool SOMETHING 2026-02-12T18:16:16.254 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-12T18:16:16.254 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-12T18:16:16.254 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-12T18:16:16.254 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:16:16.483 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=6 2026-02-12T18:16:16.483 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:16:16.483 INFO:tasks.workunit.client.0.trial116.stderr:7 2026-02-12T18:16:16.483 INFO:tasks.workunit.client.0.trial116.stderr:9' 2026-02-12T18:16:16.483 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 6 0 7 9 2026-02-12T18:16:16.483 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('6' '0' '7' '9') 2026-02-12T18:16:16.484 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-12T18:16:16.484 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=3 2026-02-12T18:16:16.484 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 9 2026-02-12T18:16:16.729 INFO:tasks.workunit.client.0.trial116.stderr:osd.9 is already out. 2026-02-12T18:16:16.737 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-12T18:16:21.738 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds ecpool SOMETHING 2026-02-12T18:16:21.738 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-12T18:16:21.738 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-12T18:16:21.738 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<9\>' 2026-02-12T18:16:21.739 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-12T18:16:21.739 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:16:21.979 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=6 2026-02-12T18:16:21.979 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:16:21.979 INFO:tasks.workunit.client.0.trial116.stderr:7 2026-02-12T18:16:21.979 INFO:tasks.workunit.client.0.trial116.stderr:5' 2026-02-12T18:16:21.979 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 6 0 7 5 2026-02-12T18:16:21.980 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool ecpool get SOMETHING td/test-erasure-code/COPY 2026-02-12T18:16:21.997 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-12T18:16:21.998 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 9 2026-02-12T18:16:22.280 INFO:tasks.workunit.client.0.trial116.stderr:osd.9 is already in. 2026-02-12T18:16:22.289 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-12T18:16:22.289 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:216: TEST_rados_put_get_jerasure: local poolname=pool-jerasure 2026-02-12T18:16:22.289 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:217: TEST_rados_put_get_jerasure: local profile=profile-jerasure 2026-02-12T18:16:22.290 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:219: TEST_rados_put_get_jerasure: ceph osd erasure-code-profile set profile-jerasure plugin=jerasure k=4 m=2 crush-failure-domain=osd 2026-02-12T18:16:22.584 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:223: TEST_rados_put_get_jerasure: create_pool pool-jerasure 12 12 erasure profile-jerasure 2026-02-12T18:16:22.584 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 12 12 erasure profile-jerasure 2026-02-12T18:16:22.967 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-jerasure' already exists 2026-02-12T18:16:22.977 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:16:23.979 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:226: TEST_rados_put_get_jerasure: rados_put_get td/test-erasure-code pool-jerasure 2026-02-12T18:16:23.979 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-12T18:16:23.979 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-jerasure 2026-02-12T18:16:23.979 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-12T18:16:23.979 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-12T18:16:23.979 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-12T18:16:23.979 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-12T18:16:23.979 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-12T18:16:23.979 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-12T18:16:23.979 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-12T18:16:23.979 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-12T18:16:23.979 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-12T18:16:23.979 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-jerasure put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-12T18:16:24.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-12T18:16:24.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-12T18:16:24.030 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-12T18:16:24.031 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-jerasure SOMETHING 2026-02-12T18:16:24.031 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:16:24.031 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-12T18:16:24.031 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-12T18:16:24.031 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:16:24.262 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-12T18:16:24.262 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:16:24.262 INFO:tasks.workunit.client.0.trial116.stderr:9 2026-02-12T18:16:24.262 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:16:24.262 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:16:24.262 INFO:tasks.workunit.client.0.trial116.stderr:10' 2026-02-12T18:16:24.262 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 10 2026-02-12T18:16:24.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('5' '1' '9' '4' '3' '10') 2026-02-12T18:16:24.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-12T18:16:24.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=5 2026-02-12T18:16:24.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 10 2026-02-12T18:16:24.544 INFO:tasks.workunit.client.0.trial116.stderr:osd.10 is already out. 2026-02-12T18:16:24.554 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-12T18:16:29.556 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-jerasure SOMETHING 2026-02-12T18:16:29.556 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:16:29.556 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-12T18:16:29.556 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<10\>' 2026-02-12T18:16:29.557 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-12T18:16:29.557 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:16:29.790 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-12T18:16:29.790 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:16:29.790 INFO:tasks.workunit.client.0.trial116.stderr:9 2026-02-12T18:16:29.790 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:16:29.790 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:16:29.791 INFO:tasks.workunit.client.0.trial116.stderr:6' 2026-02-12T18:16:29.791 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 6 2026-02-12T18:16:29.791 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-12T18:16:29.808 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-12T18:16:29.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 10 2026-02-12T18:16:30.098 INFO:tasks.workunit.client.0.trial116.stderr:osd.10 is already in. 2026-02-12T18:16:30.106 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-12T18:16:30.108 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:227: TEST_rados_put_get_jerasure: rados_osds_out_in td/test-erasure-code pool-jerasure 2026-02-12T18:16:30.108 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:104: rados_osds_out_in: local dir=td/test-erasure-code 2026-02-12T18:16:30.108 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:105: rados_osds_out_in: local poolname=pool-jerasure 2026-02-12T18:16:30.109 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:106: rados_osds_out_in: local objname=SOMETHING 2026-02-12T18:16:30.109 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-12T18:16:30.109 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 FFFF 2026-02-12T18:16:30.109 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-12T18:16:30.109 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 GGGG 2026-02-12T18:16:30.109 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-12T18:16:30.109 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 HHHH 2026-02-12T18:16:30.109 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-12T18:16:30.109 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 IIII 2026-02-12T18:16:30.109 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:116: rados_osds_out_in: rados --pool pool-jerasure put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-12T18:16:30.880 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:117: rados_osds_out_in: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-12T18:16:30.897 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:118: rados_osds_out_in: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-12T18:16:30.898 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:119: rados_osds_out_in: rm td/test-erasure-code/COPY 2026-02-12T18:16:30.899 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:127: rados_osds_out_in: wait_for_clean 2026-02-12T18:16:30.900 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:16:30.900 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:16:30.900 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:16:30.900 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:16:30.901 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:16:30.901 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:16:30.901 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:16:30.901 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:16:30.901 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:16:30.981 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:16:30.982 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:16:30.982 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:16:30.982 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:16:30.982 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:16:30.982 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:16:31.207 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:16:31.208 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:16:31.208 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:16:31.208 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:16:31.208 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:16:31.208 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:16:31.208 INFO:tasks.workunit.client.0.trial116.stderr:6 2026-02-12T18:16:31.208 INFO:tasks.workunit.client.0.trial116.stderr:7 2026-02-12T18:16:31.208 INFO:tasks.workunit.client.0.trial116.stderr:8 2026-02-12T18:16:31.208 INFO:tasks.workunit.client.0.trial116.stderr:9 2026-02-12T18:16:31.208 INFO:tasks.workunit.client.0.trial116.stderr:10' 2026-02-12T18:16:31.209 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:16:31.209 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:31.209 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:16:31.272 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836514 2026-02-12T18:16:31.272 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836514 2026-02-12T18:16:31.272 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514' 2026-02-12T18:16:31.272 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:31.272 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:16:31.338 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705698 2026-02-12T18:16:31.338 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705698 2026-02-12T18:16:31.338 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514 1-38654705698' 2026-02-12T18:16:31.338 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:31.338 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:16:31.405 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574881 2026-02-12T18:16:31.405 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574881 2026-02-12T18:16:31.405 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514 1-38654705698 2-55834574881' 2026-02-12T18:16:31.405 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:31.405 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:16:31.469 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=317827579926 2026-02-12T18:16:31.469 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 317827579926 2026-02-12T18:16:31.469 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514 1-38654705698 2-55834574881 3-317827579926' 2026-02-12T18:16:31.469 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:31.469 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-12T18:16:31.533 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182432 2026-02-12T18:16:31.534 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182432 2026-02-12T18:16:31.534 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514 1-38654705698 2-55834574881 3-317827579926 4-107374182432' 2026-02-12T18:16:31.534 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:31.534 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-12T18:16:31.597 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=339302416402 2026-02-12T18:16:31.597 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 339302416402 2026-02-12T18:16:31.597 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514 1-38654705698 2-55834574881 3-317827579926 4-107374182432 5-339302416402' 2026-02-12T18:16:31.597 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:31.597 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-12T18:16:31.662 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=395136991246 2026-02-12T18:16:31.662 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 395136991246 2026-02-12T18:16:31.662 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514 1-38654705698 2-55834574881 3-317827579926 4-107374182432 5-339302416402 6-395136991246' 2026-02-12T18:16:31.662 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:31.663 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-12T18:16:31.727 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724575 2026-02-12T18:16:31.727 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724575 2026-02-12T18:16:31.727 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514 1-38654705698 2-55834574881 3-317827579926 4-107374182432 5-339302416402 6-395136991246 7-167503724575' 2026-02-12T18:16:31.727 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:31.727 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-12T18:16:31.791 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=188978561054 2026-02-12T18:16:31.791 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 188978561054 2026-02-12T18:16:31.791 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514 1-38654705698 2-55834574881 3-317827579926 4-107374182432 5-339302416402 6-395136991246 7-167503724575 8-188978561054' 2026-02-12T18:16:31.792 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:31.792 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-12T18:16:31.855 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=420906795019 2026-02-12T18:16:31.855 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 420906795019 2026-02-12T18:16:31.855 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514 1-38654705698 2-55834574881 3-317827579926 4-107374182432 5-339302416402 6-395136991246 7-167503724575 8-188978561054 9-420906795019' 2026-02-12T18:16:31.855 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:31.856 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-12T18:16:31.919 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=227633266718 2026-02-12T18:16:31.920 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 227633266718 2026-02-12T18:16:31.920 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836514 1-38654705698 2-55834574881 3-317827579926 4-107374182432 5-339302416402 6-395136991246 7-167503724575 8-188978561054 9-420906795019 10-227633266718' 2026-02-12T18:16:31.920 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:31.920 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836514 2026-02-12T18:16:31.920 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:31.921 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:16:31.922 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836514 2026-02-12T18:16:31.922 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:31.923 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836514 2026-02-12T18:16:31.923 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836514' 2026-02-12T18:16:31.924 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.0 seq 21474836514 2026-02-12T18:16:31.924 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:16:32.148 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836514 -lt 21474836514 2026-02-12T18:16:32.148 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:32.149 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705698 2026-02-12T18:16:32.149 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:32.150 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:16:32.151 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705698 2026-02-12T18:16:32.151 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:32.152 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705698 2026-02-12T18:16:32.152 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705698' 2026-02-12T18:16:32.152 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.1 seq 38654705698 2026-02-12T18:16:32.152 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:16:32.377 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705698 -lt 38654705698 2026-02-12T18:16:32.377 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:32.377 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574881 2026-02-12T18:16:32.377 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:32.378 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:16:32.379 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574881 2026-02-12T18:16:32.379 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:32.380 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574881 2026-02-12T18:16:32.380 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574881' 2026-02-12T18:16:32.380 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.2 seq 55834574881 2026-02-12T18:16:32.381 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:16:32.606 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574881 -lt 55834574881 2026-02-12T18:16:32.606 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:32.607 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-317827579926 2026-02-12T18:16:32.607 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:32.608 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:16:32.609 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-317827579926 2026-02-12T18:16:32.609 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:32.610 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=317827579926 2026-02-12T18:16:32.610 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 317827579926' 2026-02-12T18:16:32.611 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.3 seq 317827579926 2026-02-12T18:16:32.611 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:16:32.834 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 317827579926 -lt 317827579926 2026-02-12T18:16:32.834 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:32.835 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-107374182432 2026-02-12T18:16:32.835 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:32.836 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-12T18:16:32.837 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-107374182432 2026-02-12T18:16:32.837 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:32.838 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182432 2026-02-12T18:16:32.838 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 107374182432' 2026-02-12T18:16:32.838 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.4 seq 107374182432 2026-02-12T18:16:32.838 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-12T18:16:33.069 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182432 -lt 107374182432 2026-02-12T18:16:33.069 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:33.070 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-339302416402 2026-02-12T18:16:33.070 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:33.070 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-12T18:16:33.071 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-339302416402 2026-02-12T18:16:33.071 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:33.072 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=339302416402 2026-02-12T18:16:33.073 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 339302416402' 2026-02-12T18:16:33.073 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.5 seq 339302416402 2026-02-12T18:16:33.073 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-12T18:16:33.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 339302416402 -lt 339302416402 2026-02-12T18:16:33.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:33.300 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-395136991246 2026-02-12T18:16:33.300 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:33.301 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-12T18:16:33.302 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-395136991246 2026-02-12T18:16:33.302 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:33.303 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=395136991246 2026-02-12T18:16:33.303 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 395136991246' 2026-02-12T18:16:33.304 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.6 seq 395136991246 2026-02-12T18:16:33.304 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-12T18:16:33.533 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 395136991246 -lt 395136991246 2026-02-12T18:16:33.533 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:33.533 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-167503724575 2026-02-12T18:16:33.534 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:33.534 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-12T18:16:33.535 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-167503724575 2026-02-12T18:16:33.535 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:33.536 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=167503724575 2026-02-12T18:16:33.536 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 167503724575' 2026-02-12T18:16:33.536 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.7 seq 167503724575 2026-02-12T18:16:33.537 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-12T18:16:33.765 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 167503724575 -lt 167503724575 2026-02-12T18:16:33.765 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:33.765 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-188978561054 2026-02-12T18:16:33.766 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:33.766 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-12T18:16:33.767 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-188978561054 2026-02-12T18:16:33.767 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:33.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=188978561054 2026-02-12T18:16:33.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 188978561054' 2026-02-12T18:16:33.769 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.8 seq 188978561054 2026-02-12T18:16:33.769 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-12T18:16:33.998 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 188978561055 -lt 188978561054 2026-02-12T18:16:33.998 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:33.999 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-420906795019 2026-02-12T18:16:33.999 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:33.999 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-12T18:16:34.000 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-420906795019 2026-02-12T18:16:34.000 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:34.001 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=420906795019 2026-02-12T18:16:34.001 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 420906795019' 2026-02-12T18:16:34.001 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.9 seq 420906795019 2026-02-12T18:16:34.002 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-12T18:16:34.228 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 420906795019 -lt 420906795019 2026-02-12T18:16:34.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:34.229 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-227633266718 2026-02-12T18:16:34.229 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:34.230 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-12T18:16:34.231 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-227633266718 2026-02-12T18:16:34.231 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:34.232 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=227633266718 2026-02-12T18:16:34.232 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 227633266718' 2026-02-12T18:16:34.232 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.10 seq 227633266718 2026-02-12T18:16:34.233 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-12T18:16:34.459 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 227633266719 -lt 227633266718 2026-02-12T18:16:34.460 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:16:34.460 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:16:34.460 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:16:34.777 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-12T18:16:34.777 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:16:34.778 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:16:34.778 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:16:34.778 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:16:34.778 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:16:34.778 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:16:34.778 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:16:35.001 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-12T18:16:35.002 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:16:35.002 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:16:35.003 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:16:35.331 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-12T18:16:35.332 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:16:35.332 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:16:35.333 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:128: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-12T18:16:35.333 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:16:35.333 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-12T18:16:35.333 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-12T18:16:35.333 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:16:35.564 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-12T18:16:35.564 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:16:35.564 INFO:tasks.workunit.client.0.trial116.stderr:9 2026-02-12T18:16:35.564 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:16:35.564 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:16:35.564 INFO:tasks.workunit.client.0.trial116.stderr:10' 2026-02-12T18:16:35.564 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 10 2026-02-12T18:16:35.565 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:128: rados_osds_out_in: local 'osds_list=5 1 9 4 3 10' 2026-02-12T18:16:35.565 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:129: rados_osds_out_in: osds=('5' '1' '9' '4' '3' '10') 2026-02-12T18:16:35.565 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:129: rados_osds_out_in: local -a osds 2026-02-12T18:16:35.565 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:130: rados_osds_out_in: for osd in 0 1 2026-02-12T18:16:35.565 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:131: rados_osds_out_in: ceph osd out 5 2026-02-12T18:16:35.816 INFO:tasks.workunit.client.0.trial116.stderr:osd.5 is already out. 2026-02-12T18:16:35.825 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:130: rados_osds_out_in: for osd in 0 1 2026-02-12T18:16:35.826 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:131: rados_osds_out_in: ceph osd out 1 2026-02-12T18:16:36.118 INFO:tasks.workunit.client.0.trial116.stderr:osd.1 is already out. 2026-02-12T18:16:36.126 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:133: rados_osds_out_in: wait_for_clean 2026-02-12T18:16:36.126 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:16:36.126 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:16:36.127 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:16:36.128 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:16:36.128 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:16:36.128 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:16:36.128 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:16:36.128 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:16:36.128 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:16:36.212 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:16:36.212 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:16:36.212 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:16:36.212 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:16:36.212 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:16:36.212 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:16:36.439 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:16:36.440 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:16:36.440 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:16:36.440 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:16:36.440 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:16:36.440 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:16:36.440 INFO:tasks.workunit.client.0.trial116.stderr:6 2026-02-12T18:16:36.440 INFO:tasks.workunit.client.0.trial116.stderr:7 2026-02-12T18:16:36.440 INFO:tasks.workunit.client.0.trial116.stderr:8 2026-02-12T18:16:36.440 INFO:tasks.workunit.client.0.trial116.stderr:9 2026-02-12T18:16:36.441 INFO:tasks.workunit.client.0.trial116.stderr:10' 2026-02-12T18:16:36.441 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:16:36.441 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:36.441 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:16:36.505 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836517 2026-02-12T18:16:36.505 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836517 2026-02-12T18:16:36.505 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517' 2026-02-12T18:16:36.506 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:36.506 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:16:36.568 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705701 2026-02-12T18:16:36.568 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705701 2026-02-12T18:16:36.568 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517 1-38654705701' 2026-02-12T18:16:36.568 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:36.569 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:16:36.633 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574884 2026-02-12T18:16:36.633 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574884 2026-02-12T18:16:36.633 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517 1-38654705701 2-55834574884' 2026-02-12T18:16:36.633 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:36.633 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:16:36.700 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=317827579929 2026-02-12T18:16:36.700 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 317827579929 2026-02-12T18:16:36.701 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517 1-38654705701 2-55834574884 3-317827579929' 2026-02-12T18:16:36.701 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:36.701 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-12T18:16:36.764 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182435 2026-02-12T18:16:36.764 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182435 2026-02-12T18:16:36.764 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517 1-38654705701 2-55834574884 3-317827579929 4-107374182435' 2026-02-12T18:16:36.764 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:36.764 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-12T18:16:36.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=339302416405 2026-02-12T18:16:36.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 339302416405 2026-02-12T18:16:36.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517 1-38654705701 2-55834574884 3-317827579929 4-107374182435 5-339302416405' 2026-02-12T18:16:36.829 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:36.829 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-12T18:16:36.892 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=395136991249 2026-02-12T18:16:36.892 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 395136991249 2026-02-12T18:16:36.892 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517 1-38654705701 2-55834574884 3-317827579929 4-107374182435 5-339302416405 6-395136991249' 2026-02-12T18:16:36.892 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:36.893 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-12T18:16:36.958 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724578 2026-02-12T18:16:36.958 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724578 2026-02-12T18:16:36.958 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517 1-38654705701 2-55834574884 3-317827579929 4-107374182435 5-339302416405 6-395136991249 7-167503724578' 2026-02-12T18:16:36.958 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:36.958 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-12T18:16:37.021 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=188978561057 2026-02-12T18:16:37.021 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 188978561057 2026-02-12T18:16:37.022 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517 1-38654705701 2-55834574884 3-317827579929 4-107374182435 5-339302416405 6-395136991249 7-167503724578 8-188978561057' 2026-02-12T18:16:37.022 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:37.022 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-12T18:16:37.084 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=420906795022 2026-02-12T18:16:37.084 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 420906795022 2026-02-12T18:16:37.084 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517 1-38654705701 2-55834574884 3-317827579929 4-107374182435 5-339302416405 6-395136991249 7-167503724578 8-188978561057 9-420906795022' 2026-02-12T18:16:37.084 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:37.084 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-12T18:16:37.147 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=227633266721 2026-02-12T18:16:37.147 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 227633266721 2026-02-12T18:16:37.147 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836517 1-38654705701 2-55834574884 3-317827579929 4-107374182435 5-339302416405 6-395136991249 7-167503724578 8-188978561057 9-420906795022 10-227633266721' 2026-02-12T18:16:37.147 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:37.148 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836517 2026-02-12T18:16:37.148 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:37.149 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:16:37.149 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836517 2026-02-12T18:16:37.149 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:37.151 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836517 2026-02-12T18:16:37.151 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836517' 2026-02-12T18:16:37.151 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.0 seq 21474836517 2026-02-12T18:16:37.151 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:16:37.383 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836515 -lt 21474836517 2026-02-12T18:16:37.383 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:16:38.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:16:38.385 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:16:38.611 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836518 -lt 21474836517 2026-02-12T18:16:38.611 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:38.612 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705701 2026-02-12T18:16:38.612 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:38.612 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:16:38.613 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705701 2026-02-12T18:16:38.613 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:38.614 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705701 2026-02-12T18:16:38.615 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705701' 2026-02-12T18:16:38.615 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.1 seq 38654705701 2026-02-12T18:16:38.615 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:16:38.839 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705701 -lt 38654705701 2026-02-12T18:16:38.839 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:38.840 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574884 2026-02-12T18:16:38.840 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:38.840 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:16:38.841 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574884 2026-02-12T18:16:38.841 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:38.843 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574884 2026-02-12T18:16:38.843 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574884' 2026-02-12T18:16:38.843 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.2 seq 55834574884 2026-02-12T18:16:38.843 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:16:39.072 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574885 -lt 55834574884 2026-02-12T18:16:39.073 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:39.073 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-317827579929 2026-02-12T18:16:39.073 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:39.074 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:16:39.075 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-317827579929 2026-02-12T18:16:39.075 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:39.076 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=317827579929 2026-02-12T18:16:39.076 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 317827579929' 2026-02-12T18:16:39.076 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.3 seq 317827579929 2026-02-12T18:16:39.077 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:16:39.306 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 317827579929 -lt 317827579929 2026-02-12T18:16:39.307 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:39.307 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-107374182435 2026-02-12T18:16:39.307 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:39.308 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-12T18:16:39.309 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-107374182435 2026-02-12T18:16:39.309 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:39.310 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182435 2026-02-12T18:16:39.310 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 107374182435' 2026-02-12T18:16:39.311 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.4 seq 107374182435 2026-02-12T18:16:39.311 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-12T18:16:39.535 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182436 -lt 107374182435 2026-02-12T18:16:39.536 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:39.536 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-339302416405 2026-02-12T18:16:39.536 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:39.537 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-12T18:16:39.538 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-339302416405 2026-02-12T18:16:39.538 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:39.539 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=339302416405 2026-02-12T18:16:39.539 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 339302416405' 2026-02-12T18:16:39.539 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.5 seq 339302416405 2026-02-12T18:16:39.539 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-12T18:16:39.764 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 339302416406 -lt 339302416405 2026-02-12T18:16:39.764 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:39.765 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-395136991249 2026-02-12T18:16:39.765 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:39.766 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-12T18:16:39.766 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-395136991249 2026-02-12T18:16:39.767 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:39.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=395136991249 2026-02-12T18:16:39.768 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.6 seq 395136991249 2026-02-12T18:16:39.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 395136991249' 2026-02-12T18:16:39.768 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-12T18:16:40.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 395136991250 -lt 395136991249 2026-02-12T18:16:40.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:40.008 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-167503724578 2026-02-12T18:16:40.008 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:40.008 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-12T18:16:40.009 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-167503724578 2026-02-12T18:16:40.009 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:40.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=167503724578 2026-02-12T18:16:40.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 167503724578' 2026-02-12T18:16:40.010 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.7 seq 167503724578 2026-02-12T18:16:40.011 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-12T18:16:40.237 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 167503724578 -lt 167503724578 2026-02-12T18:16:40.237 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:40.238 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-188978561057 2026-02-12T18:16:40.238 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:40.238 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-12T18:16:40.239 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-188978561057 2026-02-12T18:16:40.239 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:40.240 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=188978561057 2026-02-12T18:16:40.240 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.8 seq 188978561057 2026-02-12T18:16:40.241 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 188978561057' 2026-02-12T18:16:40.241 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-12T18:16:40.467 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 188978561058 -lt 188978561057 2026-02-12T18:16:40.467 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:40.468 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-420906795022 2026-02-12T18:16:40.468 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:40.468 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-12T18:16:40.469 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-420906795022 2026-02-12T18:16:40.469 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:40.470 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=420906795022 2026-02-12T18:16:40.470 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 420906795022' 2026-02-12T18:16:40.470 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.9 seq 420906795022 2026-02-12T18:16:40.471 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-12T18:16:40.698 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 420906795022 -lt 420906795022 2026-02-12T18:16:40.699 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:40.699 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-227633266721 2026-02-12T18:16:40.699 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:40.700 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-12T18:16:40.701 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-227633266721 2026-02-12T18:16:40.701 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:40.702 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=227633266721 2026-02-12T18:16:40.702 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 227633266721' 2026-02-12T18:16:40.702 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.10 seq 227633266721 2026-02-12T18:16:40.703 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-12T18:16:40.926 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 227633266722 -lt 227633266721 2026-02-12T18:16:40.927 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:16:40.927 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:16:40.927 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:16:41.252 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-12T18:16:41.252 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:16:41.253 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:16:41.254 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:16:41.254 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:16:41.254 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:16:41.254 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:16:41.254 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:16:41.479 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-12T18:16:41.480 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:16:41.480 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:16:41.481 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:16:41.793 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-12T18:16:41.793 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:16:41.793 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:16:41.793 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:137: rados_osds_out_in: for osd in 0 1 2026-02-12T18:16:41.793 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-12T18:16:41.794 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:16:41.794 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-12T18:16:41.794 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: grep '\<5\>' 2026-02-12T18:16:41.794 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-12T18:16:41.794 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:16:42.021 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=2 2026-02-12T18:16:42.022 INFO:tasks.workunit.client.0.trial116.stderr:7 2026-02-12T18:16:42.022 INFO:tasks.workunit.client.0.trial116.stderr:9 2026-02-12T18:16:42.022 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:16:42.022 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:16:42.022 INFO:tasks.workunit.client.0.trial116.stderr:10' 2026-02-12T18:16:42.022 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 2 7 9 4 3 10 2026-02-12T18:16:42.022 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:137: rados_osds_out_in: for osd in 0 1 2026-02-12T18:16:42.023 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-12T18:16:42.023 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:16:42.023 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-12T18:16:42.023 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: grep '\<1\>' 2026-02-12T18:16:42.024 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-12T18:16:42.024 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:16:42.253 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=2 2026-02-12T18:16:42.253 INFO:tasks.workunit.client.0.trial116.stderr:7 2026-02-12T18:16:42.253 INFO:tasks.workunit.client.0.trial116.stderr:9 2026-02-12T18:16:42.254 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:16:42.254 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:16:42.254 INFO:tasks.workunit.client.0.trial116.stderr:10' 2026-02-12T18:16:42.254 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 2 7 9 4 3 10 2026-02-12T18:16:42.254 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:140: rados_osds_out_in: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-12T18:16:42.271 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:141: rados_osds_out_in: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-12T18:16:42.272 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:147: rados_osds_out_in: for osd in 0 1 2026-02-12T18:16:42.272 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:148: rados_osds_out_in: ceph osd in 5 2026-02-12T18:16:42.504 INFO:tasks.workunit.client.0.trial116.stderr:osd.5 is already in. 2026-02-12T18:16:42.512 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:147: rados_osds_out_in: for osd in 0 1 2026-02-12T18:16:42.513 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:148: rados_osds_out_in: ceph osd in 1 2026-02-12T18:16:42.808 INFO:tasks.workunit.client.0.trial116.stderr:osd.1 is already in. 2026-02-12T18:16:42.817 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:150: rados_osds_out_in: wait_for_clean 2026-02-12T18:16:42.817 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:16:42.817 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:16:42.817 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:16:42.818 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:16:42.818 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:16:42.818 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:16:42.818 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:16:42.818 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:16:42.818 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:16:42.900 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:16:42.900 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:16:42.900 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:16:42.900 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:16:42.900 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:16:42.900 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:16:43.121 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:16:43.122 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:16:43.122 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:16:43.122 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:16:43.122 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:16:43.122 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:16:43.122 INFO:tasks.workunit.client.0.trial116.stderr:6 2026-02-12T18:16:43.122 INFO:tasks.workunit.client.0.trial116.stderr:7 2026-02-12T18:16:43.122 INFO:tasks.workunit.client.0.trial116.stderr:8 2026-02-12T18:16:43.122 INFO:tasks.workunit.client.0.trial116.stderr:9 2026-02-12T18:16:43.122 INFO:tasks.workunit.client.0.trial116.stderr:10' 2026-02-12T18:16:43.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:16:43.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:43.122 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:16:43.186 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836521 2026-02-12T18:16:43.187 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836521 2026-02-12T18:16:43.187 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836521' 2026-02-12T18:16:43.187 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:43.187 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:16:43.251 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705704 2026-02-12T18:16:43.251 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705704 2026-02-12T18:16:43.251 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836521 1-38654705704' 2026-02-12T18:16:43.251 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:43.252 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:16:43.316 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574888 2026-02-12T18:16:43.316 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574888 2026-02-12T18:16:43.316 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836521 1-38654705704 2-55834574888' 2026-02-12T18:16:43.316 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:43.316 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:16:43.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=317827579932 2026-02-12T18:16:43.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 317827579932 2026-02-12T18:16:43.380 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836521 1-38654705704 2-55834574888 3-317827579932' 2026-02-12T18:16:43.380 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:43.380 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-12T18:16:43.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182439 2026-02-12T18:16:43.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182439 2026-02-12T18:16:43.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836521 1-38654705704 2-55834574888 3-317827579932 4-107374182439' 2026-02-12T18:16:43.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:43.443 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-12T18:16:43.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=339302416409 2026-02-12T18:16:43.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 339302416409 2026-02-12T18:16:43.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836521 1-38654705704 2-55834574888 3-317827579932 4-107374182439 5-339302416409' 2026-02-12T18:16:43.509 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:43.509 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-12T18:16:43.573 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=395136991253 2026-02-12T18:16:43.573 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 395136991253 2026-02-12T18:16:43.573 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836521 1-38654705704 2-55834574888 3-317827579932 4-107374182439 5-339302416409 6-395136991253' 2026-02-12T18:16:43.573 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:43.573 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-12T18:16:43.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724581 2026-02-12T18:16:43.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724581 2026-02-12T18:16:43.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836521 1-38654705704 2-55834574888 3-317827579932 4-107374182439 5-339302416409 6-395136991253 7-167503724581' 2026-02-12T18:16:43.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:43.638 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-12T18:16:43.702 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=188978561061 2026-02-12T18:16:43.703 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 188978561061 2026-02-12T18:16:43.703 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836521 1-38654705704 2-55834574888 3-317827579932 4-107374182439 5-339302416409 6-395136991253 7-167503724581 8-188978561061' 2026-02-12T18:16:43.703 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:43.703 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-12T18:16:43.767 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=420906795025 2026-02-12T18:16:43.767 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 420906795025 2026-02-12T18:16:43.767 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836521 1-38654705704 2-55834574888 3-317827579932 4-107374182439 5-339302416409 6-395136991253 7-167503724581 8-188978561061 9-420906795025' 2026-02-12T18:16:43.767 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:16:43.767 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-12T18:16:43.831 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=227633266725 2026-02-12T18:16:43.831 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 227633266725 2026-02-12T18:16:43.831 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836521 1-38654705704 2-55834574888 3-317827579932 4-107374182439 5-339302416409 6-395136991253 7-167503724581 8-188978561061 9-420906795025 10-227633266725' 2026-02-12T18:16:43.832 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:43.832 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836521 2026-02-12T18:16:43.832 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:43.833 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:16:43.834 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836521 2026-02-12T18:16:43.834 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:43.835 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836521 2026-02-12T18:16:43.835 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836521' 2026-02-12T18:16:43.835 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.0 seq 21474836521 2026-02-12T18:16:43.835 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:16:44.068 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836521 -lt 21474836521 2026-02-12T18:16:44.068 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:44.069 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705704 2026-02-12T18:16:44.069 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:44.070 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:16:44.071 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705704 2026-02-12T18:16:44.071 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:44.072 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705704 2026-02-12T18:16:44.072 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705704' 2026-02-12T18:16:44.072 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.1 seq 38654705704 2026-02-12T18:16:44.072 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:16:44.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705704 -lt 38654705704 2026-02-12T18:16:44.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:44.298 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574888 2026-02-12T18:16:44.298 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:44.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:16:44.299 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574888 2026-02-12T18:16:44.300 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:44.300 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574888 2026-02-12T18:16:44.301 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.2 seq 55834574888 2026-02-12T18:16:44.301 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574888' 2026-02-12T18:16:44.301 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:16:44.532 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574888 -lt 55834574888 2026-02-12T18:16:44.532 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:44.532 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-317827579932 2026-02-12T18:16:44.533 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:44.533 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:16:44.534 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-317827579932 2026-02-12T18:16:44.534 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:44.535 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=317827579932 2026-02-12T18:16:44.535 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.3 seq 317827579932 2026-02-12T18:16:44.535 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 317827579932' 2026-02-12T18:16:44.536 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:16:44.765 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 317827579932 -lt 317827579932 2026-02-12T18:16:44.766 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:44.766 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-107374182439 2026-02-12T18:16:44.766 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:44.767 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-12T18:16:44.768 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-107374182439 2026-02-12T18:16:44.768 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:44.769 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182439 2026-02-12T18:16:44.769 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 107374182439' 2026-02-12T18:16:44.769 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.4 seq 107374182439 2026-02-12T18:16:44.769 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-12T18:16:44.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182439 -lt 107374182439 2026-02-12T18:16:44.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:44.995 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-339302416409 2026-02-12T18:16:44.995 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:44.996 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-12T18:16:44.997 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-339302416409 2026-02-12T18:16:44.997 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:44.998 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=339302416409 2026-02-12T18:16:44.998 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 339302416409' 2026-02-12T18:16:44.998 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.5 seq 339302416409 2026-02-12T18:16:44.999 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-12T18:16:45.226 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 339302416409 -lt 339302416409 2026-02-12T18:16:45.226 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:45.226 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-395136991253 2026-02-12T18:16:45.226 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:45.227 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-12T18:16:45.228 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-395136991253 2026-02-12T18:16:45.228 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:45.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=395136991253 2026-02-12T18:16:45.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 395136991253' 2026-02-12T18:16:45.229 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.6 seq 395136991253 2026-02-12T18:16:45.230 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-12T18:16:45.461 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 395136991253 -lt 395136991253 2026-02-12T18:16:45.461 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:45.461 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-167503724581 2026-02-12T18:16:45.461 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:45.462 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-12T18:16:45.463 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-167503724581 2026-02-12T18:16:45.463 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:45.464 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=167503724581 2026-02-12T18:16:45.464 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 167503724581' 2026-02-12T18:16:45.464 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.7 seq 167503724581 2026-02-12T18:16:45.464 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-12T18:16:45.689 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 167503724581 -lt 167503724581 2026-02-12T18:16:45.689 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:45.689 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-188978561061 2026-02-12T18:16:45.689 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:45.690 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-12T18:16:45.691 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-188978561061 2026-02-12T18:16:45.691 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:45.692 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=188978561061 2026-02-12T18:16:45.692 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.8 seq 188978561061 2026-02-12T18:16:45.692 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 188978561061' 2026-02-12T18:16:45.692 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-12T18:16:45.929 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 188978561061 -lt 188978561061 2026-02-12T18:16:45.929 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:45.929 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 9-420906795025 2026-02-12T18:16:45.929 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:45.930 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=9 2026-02-12T18:16:45.931 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-420906795025 2026-02-12T18:16:45.931 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:45.932 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=420906795025 2026-02-12T18:16:45.932 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.9 seq 420906795025' 2026-02-12T18:16:45.932 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.9 seq 420906795025 2026-02-12T18:16:45.933 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-12T18:16:46.159 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 420906795026 -lt 420906795025 2026-02-12T18:16:46.159 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:16:46.159 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 10-227633266725 2026-02-12T18:16:46.159 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:16:46.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=10 2026-02-12T18:16:46.161 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-227633266725 2026-02-12T18:16:46.161 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:16:46.162 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=227633266725 2026-02-12T18:16:46.162 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.10 seq 227633266725 2026-02-12T18:16:46.162 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.10 seq 227633266725' 2026-02-12T18:16:46.162 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-12T18:16:46.393 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 227633266725 -lt 227633266725 2026-02-12T18:16:46.393 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:16:46.394 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:16:46.394 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:16:46.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-12T18:16:46.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:16:46.708 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:16:46.708 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:16:46.708 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:16:46.708 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:16:46.708 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:16:46.709 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:16:46.938 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=24 2026-02-12T18:16:46.938 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:16:46.938 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:16:46.938 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:16:47.257 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 24 = 28 2026-02-12T18:16:47.257 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 24 '!=' -1 2026-02-12T18:16:47.257 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-12T18:16:47.257 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=24 2026-02-12T18:16:47.257 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-12T18:16:47.258 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-12T18:16:47.358 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-12T18:16:47.359 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:16:47.360 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:16:47.360 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:16:47.360 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:16:47.360 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:16:47.360 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:16:47.360 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:16:47.584 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=24 2026-02-12T18:16:47.584 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:16:47.585 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:16:47.585 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:16:47.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 24 = 28 2026-02-12T18:16:47.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 24 '!=' 24 2026-02-12T18:16:47.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-12T18:16:47.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-12T18:16:47.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-12T18:16:47.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-12T18:16:47.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-12T18:16:47.904 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-12T18:16:47.904 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-12T18:16:48.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=3044 2026-02-12T18:16:48.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test 3044 '!=' null 2026-02-12T18:16:48.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1668: wait_for_clean: loop=0 2026-02-12T18:16:48.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-12T18:16:48.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-12T18:16:48.317 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-12T18:16:48.318 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:16:48.318 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:16:48.319 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:16:48.319 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:16:48.319 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:16:48.319 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:16:48.320 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:16:48.549 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-12T18:16:48.550 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:16:48.551 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:16:48.551 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:16:48.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-12T18:16:48.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:16:48.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:16:48.904 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:151: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-12T18:16:48.904 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:16:48.904 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-12T18:16:48.905 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-12T18:16:48.905 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:16:49.138 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-12T18:16:49.138 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:16:49.138 INFO:tasks.workunit.client.0.trial116.stderr:9 2026-02-12T18:16:49.138 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:16:49.138 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:16:49.138 INFO:tasks.workunit.client.0.trial116.stderr:10' 2026-02-12T18:16:49.138 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 10 2026-02-12T18:16:49.138 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:151: rados_osds_out_in: test '5 1 9 4 3 10' = '5 1 9 4 3 10' 2026-02-12T18:16:49.139 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:152: rados_osds_out_in: rm td/test-erasure-code/ORIGINAL 2026-02-12T18:16:49.139 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:229: TEST_rados_put_get_jerasure: delete_pool pool-jerasure 2026-02-12T18:16:49.139 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-jerasure 2026-02-12T18:16:49.140 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-12T18:16:49.382 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-jerasure' does not exist 2026-02-12T18:16:49.391 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:230: TEST_rados_put_get_jerasure: ceph osd erasure-code-profile rm profile-jerasure 2026-02-12T18:16:49.674 INFO:tasks.workunit.client.0.trial116.stderr:erasure-code-profile profile-jerasure does not exist 2026-02-12T18:16:49.682 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-12T18:16:49.682 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_lrc_advanced td/test-erasure-code 2026-02-12T18:16:49.682 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:156: TEST_rados_put_get_lrc_advanced: local dir=td/test-erasure-code 2026-02-12T18:16:49.683 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:157: TEST_rados_put_get_lrc_advanced: local poolname=pool-lrc-a 2026-02-12T18:16:49.683 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:158: TEST_rados_put_get_lrc_advanced: local profile=profile-lrc-a 2026-02-12T18:16:49.683 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:160: TEST_rados_put_get_lrc_advanced: ceph osd erasure-code-profile set profile-lrc-a plugin=lrc mapping=DD_ 'crush-steps=[ [ "chooseleaf", "osd", 0 ] ]' 'layers=[ [ "DDc", "" ] ]' 2026-02-12T18:16:49.974 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:165: TEST_rados_put_get_lrc_advanced: create_pool pool-lrc-a 12 12 erasure profile-lrc-a 2026-02-12T18:16:49.974 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-lrc-a 12 12 erasure profile-lrc-a 2026-02-12T18:16:50.353 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-lrc-a' already exists 2026-02-12T18:16:50.363 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:16:51.365 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:168: TEST_rados_put_get_lrc_advanced: rados_put_get td/test-erasure-code pool-lrc-a 2026-02-12T18:16:51.366 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-12T18:16:51.366 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-lrc-a 2026-02-12T18:16:51.366 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-12T18:16:51.366 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-12T18:16:51.366 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-12T18:16:51.366 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-12T18:16:51.366 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-12T18:16:51.366 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-12T18:16:51.366 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-12T18:16:51.367 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-12T18:16:51.367 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-12T18:16:51.367 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-lrc-a put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-12T18:16:51.383 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-lrc-a get SOMETHING td/test-erasure-code/COPY 2026-02-12T18:16:51.400 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-12T18:16:51.401 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-12T18:16:51.402 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-lrc-a SOMETHING 2026-02-12T18:16:51.402 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc-a 2026-02-12T18:16:51.402 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-12T18:16:51.403 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc-a SOMETHING 2026-02-12T18:16:51.403 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:16:51.635 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-12T18:16:51.635 INFO:tasks.workunit.client.0.trial116.stderr:9 2026-02-12T18:16:51.636 INFO:tasks.workunit.client.0.trial116.stderr:3' 2026-02-12T18:16:51.636 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 9 3 2026-02-12T18:16:51.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('5' '9' '3') 2026-02-12T18:16:51.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-12T18:16:51.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=2 2026-02-12T18:16:51.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 3 2026-02-12T18:16:51.885 INFO:tasks.workunit.client.0.trial116.stderr:osd.3 is already out. 2026-02-12T18:16:51.894 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-12T18:16:56.896 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-lrc-a SOMETHING 2026-02-12T18:16:56.896 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc-a 2026-02-12T18:16:56.896 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-12T18:16:56.896 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<3\>' 2026-02-12T18:16:56.897 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc-a SOMETHING 2026-02-12T18:16:56.897 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:16:57.123 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-12T18:16:57.123 INFO:tasks.workunit.client.0.trial116.stderr:9 2026-02-12T18:16:57.123 INFO:tasks.workunit.client.0.trial116.stderr:7' 2026-02-12T18:16:57.123 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 9 7 2026-02-12T18:16:57.123 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-lrc-a get SOMETHING td/test-erasure-code/COPY 2026-02-12T18:16:57.140 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-12T18:16:57.141 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 3 2026-02-12T18:16:57.372 INFO:tasks.workunit.client.0.trial116.stderr:osd.3 is already in. 2026-02-12T18:16:57.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-12T18:16:57.382 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:170: TEST_rados_put_get_lrc_advanced: delete_pool pool-lrc-a 2026-02-12T18:16:57.382 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-lrc-a 2026-02-12T18:16:57.382 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-lrc-a pool-lrc-a --yes-i-really-really-mean-it 2026-02-12T18:16:57.673 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-lrc-a' does not exist 2026-02-12T18:16:57.681 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:171: TEST_rados_put_get_lrc_advanced: ceph osd erasure-code-profile rm profile-lrc-a 2026-02-12T18:16:57.962 INFO:tasks.workunit.client.0.trial116.stderr:erasure-code-profile profile-lrc-a does not exist 2026-02-12T18:16:57.970 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-12T18:16:57.971 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_lrc_kml td/test-erasure-code 2026-02-12T18:16:57.971 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:175: TEST_rados_put_get_lrc_kml: local dir=td/test-erasure-code 2026-02-12T18:16:57.971 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:176: TEST_rados_put_get_lrc_kml: local poolname=pool-lrc 2026-02-12T18:16:57.971 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:177: TEST_rados_put_get_lrc_kml: local profile=profile-lrc 2026-02-12T18:16:57.971 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:179: TEST_rados_put_get_lrc_kml: ceph osd erasure-code-profile set profile-lrc plugin=lrc k=4 m=2 l=3 crush-failure-domain=osd 2026-02-12T18:16:58.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:183: TEST_rados_put_get_lrc_kml: create_pool pool-lrc 12 12 erasure profile-lrc 2026-02-12T18:16:58.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-lrc 12 12 erasure profile-lrc 2026-02-12T18:16:58.637 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-lrc' already exists 2026-02-12T18:16:58.646 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:16:59.647 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:186: TEST_rados_put_get_lrc_kml: rados_put_get td/test-erasure-code pool-lrc 2026-02-12T18:16:59.647 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-12T18:16:59.647 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-lrc 2026-02-12T18:16:59.647 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-12T18:16:59.648 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-12T18:16:59.648 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-12T18:16:59.648 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-12T18:16:59.648 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-12T18:16:59.648 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-12T18:16:59.648 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-12T18:16:59.648 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-12T18:16:59.648 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-12T18:16:59.648 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-lrc put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-12T18:16:59.668 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-lrc get SOMETHING td/test-erasure-code/COPY 2026-02-12T18:16:59.684 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-12T18:16:59.685 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-12T18:16:59.687 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-lrc SOMETHING 2026-02-12T18:16:59.687 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc 2026-02-12T18:16:59.687 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-12T18:16:59.687 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc SOMETHING 2026-02-12T18:16:59.687 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:16:59.920 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=8 2026-02-12T18:16:59.921 INFO:tasks.workunit.client.0.trial116.stderr:6 2026-02-12T18:16:59.921 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:16:59.921 INFO:tasks.workunit.client.0.trial116.stderr:10 2026-02-12T18:16:59.921 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:16:59.921 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:16:59.921 INFO:tasks.workunit.client.0.trial116.stderr:9 2026-02-12T18:16:59.921 INFO:tasks.workunit.client.0.trial116.stderr:7' 2026-02-12T18:16:59.921 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 8 6 0 10 5 3 9 7 2026-02-12T18:16:59.921 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('8' '6' '0' '10' '5' '3' '9' '7') 2026-02-12T18:16:59.921 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-12T18:16:59.921 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=7 2026-02-12T18:16:59.921 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 7 2026-02-12T18:17:00.202 INFO:tasks.workunit.client.0.trial116.stderr:osd.7 is already out. 2026-02-12T18:17:00.211 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-12T18:17:05.213 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-lrc SOMETHING 2026-02-12T18:17:05.214 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc 2026-02-12T18:17:05.214 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-12T18:17:05.214 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<7\>' 2026-02-12T18:17:05.214 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc SOMETHING 2026-02-12T18:17:05.215 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:17:05.441 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=8 2026-02-12T18:17:05.442 INFO:tasks.workunit.client.0.trial116.stderr:6 2026-02-12T18:17:05.442 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:17:05.442 INFO:tasks.workunit.client.0.trial116.stderr:10 2026-02-12T18:17:05.442 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:17:05.442 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:17:05.442 INFO:tasks.workunit.client.0.trial116.stderr:9 2026-02-12T18:17:05.442 INFO:tasks.workunit.client.0.trial116.stderr:4' 2026-02-12T18:17:05.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 8 6 0 10 5 3 9 4 2026-02-12T18:17:05.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-lrc get SOMETHING td/test-erasure-code/COPY 2026-02-12T18:17:05.459 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-12T18:17:05.460 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 7 2026-02-12T18:17:05.691 INFO:tasks.workunit.client.0.trial116.stderr:osd.7 is already in. 2026-02-12T18:17:05.700 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-12T18:17:05.701 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:188: TEST_rados_put_get_lrc_kml: delete_pool pool-lrc 2026-02-12T18:17:05.702 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-lrc 2026-02-12T18:17:05.702 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-lrc pool-lrc --yes-i-really-really-mean-it 2026-02-12T18:17:05.996 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-lrc' does not exist 2026-02-12T18:17:06.005 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:189: TEST_rados_put_get_lrc_kml: ceph osd erasure-code-profile rm profile-lrc 2026-02-12T18:17:06.284 INFO:tasks.workunit.client.0.trial116.stderr:erasure-code-profile profile-lrc does not exist 2026-02-12T18:17:06.292 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-12T18:17:06.292 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_shec td/test-erasure-code 2026-02-12T18:17:06.293 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:234: TEST_rados_put_get_shec: local dir=td/test-erasure-code 2026-02-12T18:17:06.293 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:236: TEST_rados_put_get_shec: local poolname=pool-shec 2026-02-12T18:17:06.293 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:237: TEST_rados_put_get_shec: local profile=profile-shec 2026-02-12T18:17:06.293 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:239: TEST_rados_put_get_shec: ceph osd erasure-code-profile set profile-shec plugin=shec k=2 m=1 c=1 crush-failure-domain=osd 2026-02-12T18:17:06.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:243: TEST_rados_put_get_shec: create_pool pool-shec 12 12 erasure profile-shec 2026-02-12T18:17:06.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-shec 12 12 erasure profile-shec 2026-02-12T18:17:06.963 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-shec' already exists 2026-02-12T18:17:06.971 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:17:07.974 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:246: TEST_rados_put_get_shec: rados_put_get td/test-erasure-code pool-shec 2026-02-12T18:17:07.974 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-12T18:17:07.974 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-shec 2026-02-12T18:17:07.974 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-12T18:17:07.974 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-12T18:17:07.974 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-12T18:17:07.974 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-12T18:17:07.975 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-12T18:17:07.975 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-12T18:17:07.975 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-12T18:17:07.975 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-12T18:17:07.975 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-12T18:17:07.975 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-shec put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-12T18:17:07.992 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-shec get SOMETHING td/test-erasure-code/COPY 2026-02-12T18:17:08.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-12T18:17:08.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-12T18:17:08.011 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-shec SOMETHING 2026-02-12T18:17:08.011 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-shec 2026-02-12T18:17:08.011 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-12T18:17:08.011 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-shec SOMETHING 2026-02-12T18:17:08.012 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:17:08.239 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=7 2026-02-12T18:17:08.239 INFO:tasks.workunit.client.0.trial116.stderr:9 2026-02-12T18:17:08.239 INFO:tasks.workunit.client.0.trial116.stderr:1' 2026-02-12T18:17:08.239 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 7 9 1 2026-02-12T18:17:08.239 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('7' '9' '1') 2026-02-12T18:17:08.240 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-12T18:17:08.240 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=2 2026-02-12T18:17:08.240 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 1 2026-02-12T18:17:08.489 INFO:tasks.workunit.client.0.trial116.stderr:osd.1 is already out. 2026-02-12T18:17:08.497 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-12T18:17:13.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-shec SOMETHING 2026-02-12T18:17:13.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-shec 2026-02-12T18:17:13.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-12T18:17:13.500 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<1\>' 2026-02-12T18:17:13.500 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-shec SOMETHING 2026-02-12T18:17:13.500 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:17:13.726 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=7 2026-02-12T18:17:13.727 INFO:tasks.workunit.client.0.trial116.stderr:9 2026-02-12T18:17:13.727 INFO:tasks.workunit.client.0.trial116.stderr:6' 2026-02-12T18:17:13.727 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 7 9 6 2026-02-12T18:17:13.727 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-shec get SOMETHING td/test-erasure-code/COPY 2026-02-12T18:17:13.745 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-12T18:17:13.746 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 1 2026-02-12T18:17:14.081 INFO:tasks.workunit.client.0.trial116.stderr:osd.1 is already in. 2026-02-12T18:17:14.089 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-12T18:17:14.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:248: TEST_rados_put_get_shec: delete_pool pool-shec 2026-02-12T18:17:14.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-shec 2026-02-12T18:17:14.091 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-shec pool-shec --yes-i-really-really-mean-it 2026-02-12T18:17:14.378 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-shec' does not exist 2026-02-12T18:17:14.386 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:249: TEST_rados_put_get_shec: ceph osd erasure-code-profile rm profile-shec 2026-02-12T18:17:14.665 INFO:tasks.workunit.client.0.trial116.stderr:erasure-code-profile profile-shec does not exist 2026-02-12T18:17:14.673 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:51: run: delete_pool ecpool 2026-02-12T18:17:14.673 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=ecpool 2026-02-12T18:17:14.673 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete ecpool ecpool --yes-i-really-really-mean-it 2026-02-12T18:17:14.950 INFO:tasks.workunit.client.0.trial116.stderr:pool 'ecpool' does not exist 2026-02-12T18:17:14.958 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:52: run: teardown td/test-erasure-code 2026-02-12T18:17:14.958 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code 2026-02-12T18:17:14.958 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:17:14.959 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code KILL 2026-02-12T18:17:14.959 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:17:14.960 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:17:14.960 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:17:14.960 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:17:14.960 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:17:15.072 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:17:15.072 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:17:15.073 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:17:15.073 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:17:15.074 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:17:15.074 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:17:15.075 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:17:15.075 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:17:15.075 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:17:15.076 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:17:15.076 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:17:15.076 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:17:15.077 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:17:15.077 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code 2026-02-12T18:17:15.133 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:17:15.133 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:17:15.133 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.49040 2026-02-12T18:17:15.133 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.49040 2026-02-12T18:17:15.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:17:15.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:17:15.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-12T18:17:15.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/test-erasure-code 0 2026-02-12T18:17:15.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code 2026-02-12T18:17:15.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-12T18:17:15.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code KILL 2026-02-12T18:17:15.136 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:17:15.136 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:17:15.136 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:17:15.136 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:17:15.136 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:17:15.137 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:17:15.137 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:17:15.137 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:17:15.137 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:17:15.138 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:17:15.138 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:17:15.139 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:17:15.139 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:17:15.139 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:17:15.140 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:17:15.140 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:17:15.140 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:17:15.141 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-12T18:17:15.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code 2026-02-12T18:17:15.143 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:17:15.143 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:17:15.143 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.49040 2026-02-12T18:17:15.143 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.49040 2026-02-12T18:17:15.144 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:17:15.145 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:17:15.145 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-12T18:17:15.145 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-12T18:17:15.145 DEBUG:teuthology.orchestra.run.trial116:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-12T18:17:15.171 INFO:tasks.workunit:Running workunit erasure-code/test-erasure-eio.sh... 2026-02-12T18:17:15.171 DEBUG:teuthology.orchestra.run.trial116:workunit test erasure-code/test-erasure-eio.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=afefaa40503a4bf024e96f97dd7f852b75f00ad6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh 2026-02-12T18:17:15.233 INFO:tasks.workunit.client.0.trial116.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-12T18:17:15.239 INFO:tasks.workunit.client.0.trial116.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-12T18:17:15.239 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-12T18:17:15.239 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-12T18:17:15.239 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-12T18:17:15.239 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-12T18:17:15.239 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-12T18:17:15.239 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-12T18:17:15.239 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-12T18:17:15.239 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-12T18:17:15.240 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/test-erasure-eio 2026-02-12T18:17:15.240 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:22: run: local dir=td/test-erasure-eio 2026-02-12T18:17:15.240 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:23: run: shift 2026-02-12T18:17:15.240 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:25: run: export CEPH_MON=127.0.0.1:7112 2026-02-12T18:17:15.240 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:25: run: CEPH_MON=127.0.0.1:7112 2026-02-12T18:17:15.240 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:26: run: export CEPH_ARGS 2026-02-12T18:17:15.240 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:27: run: uuidgen 2026-02-12T18:17:15.240 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:27: run: CEPH_ARGS+='--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none ' 2026-02-12T18:17:15.240 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:28: run: CEPH_ARGS+='--mon-host=127.0.0.1:7112 ' 2026-02-12T18:17:15.240 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:29: run: CEPH_ARGS+='--osd_mclock_override_recovery_settings=true ' 2026-02-12T18:17:15.240 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:31: run: set 2026-02-12T18:17:15.241 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:31: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-12T18:17:15.243 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:31: run: local 'funcs=TEST_ec_backfill_unfound 2026-02-12T18:17:15.243 INFO:tasks.workunit.client.0.trial116.stderr:TEST_ec_object_attr_read_error 2026-02-12T18:17:15.243 INFO:tasks.workunit.client.0.trial116.stderr:TEST_ec_recovery_multiple_errors 2026-02-12T18:17:15.243 INFO:tasks.workunit.client.0.trial116.stderr:TEST_ec_recovery_multiple_objects 2026-02-12T18:17:15.243 INFO:tasks.workunit.client.0.trial116.stderr:TEST_ec_recovery_multiple_objects_eio 2026-02-12T18:17:15.243 INFO:tasks.workunit.client.0.trial116.stderr:TEST_ec_recovery_unfound 2026-02-12T18:17:15.243 INFO:tasks.workunit.client.0.trial116.stderr:TEST_ec_single_recovery_error 2026-02-12T18:17:15.243 INFO:tasks.workunit.client.0.trial116.stderr:TEST_rados_get_bad_size_shard_0 2026-02-12T18:17:15.243 INFO:tasks.workunit.client.0.trial116.stderr:TEST_rados_get_bad_size_shard_1 2026-02-12T18:17:15.244 INFO:tasks.workunit.client.0.trial116.stderr:TEST_rados_get_subread_eio_shard_0 2026-02-12T18:17:15.244 INFO:tasks.workunit.client.0.trial116.stderr:TEST_rados_get_subread_eio_shard_1 2026-02-12T18:17:15.244 INFO:tasks.workunit.client.0.trial116.stderr:TEST_rados_get_subread_missing 2026-02-12T18:17:15.244 INFO:tasks.workunit.client.0.trial116.stderr:TEST_rados_get_with_subreadall_eio_shard_0 2026-02-12T18:17:15.244 INFO:tasks.workunit.client.0.trial116.stderr:TEST_rados_get_with_subreadall_eio_shard_1' 2026-02-12T18:17:15.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-12T18:17:15.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-12T18:17:15.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-12T18:17:15.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-12T18:17:15.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-12T18:17:15.245 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:17:15.245 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-12T18:17:15.245 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:17:15.245 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:17:15.245 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:17:15.245 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:17:15.245 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:17:15.245 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:17:15.245 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:17:15.247 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:17:15.247 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:17:15.247 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:17:15.247 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:17:15.247 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:17:15.247 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:17:15.247 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:17:15.247 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:17:15.248 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:17:15.248 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:17:15.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:17:15.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-12T18:17:15.249 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:17:15.249 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:17:15.249 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:17:15.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76821 2026-02-12T18:17:15.250 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:17:15.250 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:17:15.250 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-12T18:17:15.251 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T18:17:15.251 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:17:15.251 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:17:15.251 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76821 2026-02-12T18:17:15.253 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T18:17:15.253 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-12T18:17:15.253 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-12T18:17:15.254 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T18:17:15.254 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-12T18:17:15.254 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-12T18:17:15.254 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-12T18:17:15.254 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T18:17:15.254 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T18:17:15.254 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T18:17:15.254 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-12T18:17:15.254 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-12T18:17:15.273 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T18:17:15.273 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:17:15.273 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:17:15.273 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:17:15.273 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:17:15.273 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:17:15.273 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:17:15.273 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T18:17:15.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T18:17:15.298 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T18:17:15.298 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:17:15.299 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:17:15.299 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T18:17:15.299 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T18:17:15.299 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:17:15.299 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:17:15.299 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:17:15.299 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:17:15.299 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:17:15.299 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:17:15.299 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:17:15.300 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:17:15.300 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76821/ceph-mon.a.asok config get fsid 2026-02-12T18:17:15.341 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T18:17:15.341 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:17:15.341 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:17:15.341 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T18:17:15.341 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T18:17:15.341 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:17:15.341 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:17:15.342 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:17:15.342 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:17:15.342 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:17:15.342 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:17:15.342 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:17:15.342 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:17:15.343 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76821/ceph-mon.a.asok config get mon_host 2026-02-12T18:17:15.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-12T18:17:15.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-12T18:17:15.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T18:17:15.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T18:17:15.386 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T18:17:15.386 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-12T18:17:15.386 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T18:17:15.488 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T18:17:15.488 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:17:15.488 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:17:15.489 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:17:15.489 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:17:15.489 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:17:15.489 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:17:15.489 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:17:15.490 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:17:15.505 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-12T18:17:15.505 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T18:17:15.600 INFO:tasks.workunit.client.0.trial116.stderr:pool 'rbd' already exists 2026-02-12T18:17:15.606 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:17:16.609 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-12T18:17:16.609 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:17:16.609 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:17:16.609 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:17:16.609 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:17:16.610 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:17:16.610 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:17:16.610 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-12T18:17:16.610 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-mon.a.asok log flush 2026-02-12T18:17:16.649 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:17:16.653 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-12T18:17:16.654 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:17:15.286+0000 7f956ce3bc00 0 load: jerasure load: lrc 2026-02-12T18:17:16.656 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_backfill_unfound td/test-erasure-eio 2026-02-12T18:17:16.656 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:531: TEST_ec_backfill_unfound: local dir=td/test-erasure-eio 2026-02-12T18:17:16.656 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:532: TEST_ec_backfill_unfound: local objname=myobject 2026-02-12T18:17:16.656 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:533: TEST_ec_backfill_unfound: local lastobj=300 2026-02-12T18:17:16.656 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:535: TEST_ec_backfill_unfound: local testobj=obj250 2026-02-12T18:17:16.656 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:537: TEST_ec_backfill_unfound: ORIG_ARGS='--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:17:16.656 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:538: TEST_ec_backfill_unfound: CEPH_ARGS+=' --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-12T18:17:16.656 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: setup_osds 5 2026-02-12T18:17:16.656 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=5 2026-02-12T18:17:16.656 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-12T18:17:16.657 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 5 - 1 2026-02-12T18:17:16.657 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 4 2026-02-12T18:17:16.660 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:17:16.660 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-12T18:17:16.660 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:17:16.660 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:17:16.661 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T18:17:16.661 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:17:16.661 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-12T18:17:16.661 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-12T18:17:16.661 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:17:16.661 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:17:16.661 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:17:16.661 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-12T18:17:16.661 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-12T18:17:16.661 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:17:16.661 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:17:16.661 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:17:16.661 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:17:16.662 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:17:16.662 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:17:16.662 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:17:16.662 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:17:16.662 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:17:16.662 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:17:16.662 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:17:16.662 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:17:16.662 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:17:16.662 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:17:16.662 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:17:16.662 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:17:16.663 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:17:16.663 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:17:16.663 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:17:16.663 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:17:16.663 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:17:16.663 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:17:16.663 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-12T18:17:16.663 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:17:16.663 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2c1efc63-f23c-4892-b3ac-3874ac70a26e 2026-02-12T18:17:16.663 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 2c1efc63-f23c-4892-b3ac-3874ac70a26e' 2026-02-12T18:17:16.664 INFO:tasks.workunit.client.0.trial116.stdout:add osd0 2c1efc63-f23c-4892-b3ac-3874ac70a26e 2026-02-12T18:17:16.664 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:17:16.671 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAsGY5pAqXOJxAAvPEV7x79TLTkAjJgQZCZQw== 2026-02-12T18:17:16.671 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAsGY5pAqXOJxAAvPEV7x79TLTkAjJgQZCZQw=="}' 2026-02-12T18:17:16.671 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2c1efc63-f23c-4892-b3ac-3874ac70a26e -i td/test-erasure-eio/0/new.json 2026-02-12T18:17:16.773 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:17:16.779 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-12T18:17:16.780 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAsGY5pAqXOJxAAvPEV7x79TLTkAjJgQZCZQw== --osd-uuid 2c1efc63-f23c-4892-b3ac-3874ac70a26e 2026-02-12T18:17:16.799 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:16.795+0000 7f66c305d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:16.800 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:16.797+0000 7f66c305d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:16.801 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:16.797+0000 7f66c305d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:16.802 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:16.798+0000 7f66c305d780 -1 bdev(0x3e90c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-12T18:17:16.802 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:16.798+0000 7f66c305d780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-12T18:17:17.238 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-12T18:17:17.238 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:17:17.239 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T18:17:17.239 INFO:tasks.workunit.client.0.trial116.stdout:adding osd0 key to auth repository 2026-02-12T18:17:17.239 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:17:17.344 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T18:17:17.344 INFO:tasks.workunit.client.0.trial116.stdout:start osd.0 2026-02-12T18:17:17.345 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:17:17.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:17:17.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:17:17.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:17:17.360 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:17.356+0000 7fe0c0a23780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:17.362 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:17.358+0000 7fe0c0a23780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:17.362 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:17.359+0000 7fe0c0a23780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:17.447 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T18:17:17.448 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:17:17.448 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T18:17:17.448 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:17:17.448 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:17:17.448 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:17:17.448 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:17:17.449 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:17:17.450 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:17:17.450 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:17:17.546 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:17:17.585 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:17.581+0000 7fe0c0a23780 -1 Falling back to public interface 2026-02-12T18:17:17.833 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:17.830+0000 7fe0c0a23780 -1 osd.0 0 log_to_monitors true 2026-02-12T18:17:18.547 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:17:18.548 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:17:18.548 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:17:18.549 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:17:18.549 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:17:18.549 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:17:18.781 INFO:tasks.workunit.client.0.trial116.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2496413607,v1:127.0.0.1:6803/2496413607] [v2:127.0.0.1:6804/2496413607,v1:127.0.0.1:6805/2496413607] exists,up 2c1efc63-f23c-4892-b3ac-3874ac70a26e 2026-02-12T18:17:18.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:17:18.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:17:18.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:17:18.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:17:18.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-12T18:17:18.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:17:18.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:17:18.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-12T18:17:18.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:17:18.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-12T18:17:18.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-12T18:17:18.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:17:18.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:17:18.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:17:18.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-12T18:17:18.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-12T18:17:18.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:17:18.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:17:18.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:17:18.782 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:17:18.782 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:17:18.782 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:17:18.783 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:17:18.783 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:17:18.783 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:17:18.783 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:17:18.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:17:18.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:17:18.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:17:18.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:17:18.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:17:18.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:17:18.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:17:18.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:17:18.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:17:18.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:17:18.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:17:18.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:17:18.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-12T18:17:18.784 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:17:18.784 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=63148f8d-1c99-4462-bea2-e0246d58ed49 2026-02-12T18:17:18.785 INFO:tasks.workunit.client.0.trial116.stdout:add osd1 63148f8d-1c99-4462-bea2-e0246d58ed49 2026-02-12T18:17:18.785 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 63148f8d-1c99-4462-bea2-e0246d58ed49' 2026-02-12T18:17:18.785 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:17:18.795 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAuGY5pXrNBLxAA3YctPPtIBo4+hVIR97/Vjg== 2026-02-12T18:17:18.795 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAuGY5pXrNBLxAA3YctPPtIBo4+hVIR97/Vjg=="}' 2026-02-12T18:17:18.795 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 63148f8d-1c99-4462-bea2-e0246d58ed49 -i td/test-erasure-eio/1/new.json 2026-02-12T18:17:19.027 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:17:19.035 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-12T18:17:19.036 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAuGY5pXrNBLxAA3YctPPtIBo4+hVIR97/Vjg== --osd-uuid 63148f8d-1c99-4462-bea2-e0246d58ed49 2026-02-12T18:17:19.054 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:19.050+0000 7fa2e3e40780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:19.055 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:19.051+0000 7fa2e3e40780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:19.056 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:19.052+0000 7fa2e3e40780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:19.056 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:19.053+0000 7fa2e3e40780 -1 bdev(0x43bec00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-12T18:17:19.056 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:19.053+0000 7fa2e3e40780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-12T18:17:19.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-12T18:17:19.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:17:19.388 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-12T18:17:19.388 INFO:tasks.workunit.client.0.trial116.stdout:adding osd1 key to auth repository 2026-02-12T18:17:19.388 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:17:19.709 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-12T18:17:19.709 INFO:tasks.workunit.client.0.trial116.stdout:start osd.1 2026-02-12T18:17:19.710 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:17:19.710 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:17:19.710 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:17:19.710 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:17:19.725 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:19.722+0000 7fe7d3e46780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:19.727 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:19.723+0000 7fe7d3e46780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:19.728 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:19.724+0000 7fe7d3e46780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:19.855 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:19.852+0000 7fe7d3e46780 -1 Falling back to public interface 2026-02-12T18:17:19.942 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-12T18:17:19.942 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:17:19.942 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T18:17:19.942 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:17:19.942 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:17:19.942 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:17:19.942 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:17:19.943 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:17:19.943 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:17:19.943 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:17:20.169 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:17:20.209 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:20.205+0000 7fe7d3e46780 -1 osd.1 0 log_to_monitors true 2026-02-12T18:17:21.170 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:17:21.171 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:17:21.171 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:17:21.172 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:17:21.172 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:17:21.172 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:17:21.399 INFO:tasks.workunit.client.0.trial116.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3680748212,v1:127.0.0.1:6811/3680748212] [v2:127.0.0.1:6812/3680748212,v1:127.0.0.1:6813/3680748212] exists,up 63148f8d-1c99-4462-bea2-e0246d58ed49 2026-02-12T18:17:21.405 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:17:21.406 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:17:21.406 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:17:21.406 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:17:21.406 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-12T18:17:21.406 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:17:21.406 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:17:21.407 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-12T18:17:21.407 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:17:21.407 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-12T18:17:21.407 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-12T18:17:21.407 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:17:21.407 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:17:21.407 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:17:21.407 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-12T18:17:21.407 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-12T18:17:21.407 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:17:21.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:17:21.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:17:21.408 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:17:21.408 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:17:21.408 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:17:21.408 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:17:21.408 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:17:21.408 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:17:21.408 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:17:21.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:17:21.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:17:21.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:17:21.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:17:21.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:17:21.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:17:21.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:17:21.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:17:21.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:17:21.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:17:21.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:17:21.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:17:21.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-12T18:17:21.409 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:17:21.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5de218c3-c7f2-4940-a6fd-2eb5d4aafaee 2026-02-12T18:17:21.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 5de218c3-c7f2-4940-a6fd-2eb5d4aafaee' 2026-02-12T18:17:21.410 INFO:tasks.workunit.client.0.trial116.stdout:add osd2 5de218c3-c7f2-4940-a6fd-2eb5d4aafaee 2026-02-12T18:17:21.410 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:17:21.416 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAxGY5pCAOZGBAA1tyJDMZT4eaorwEGMcYe5w== 2026-02-12T18:17:21.416 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAxGY5pCAOZGBAA1tyJDMZT4eaorwEGMcYe5w=="}' 2026-02-12T18:17:21.416 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5de218c3-c7f2-4940-a6fd-2eb5d4aafaee -i td/test-erasure-eio/2/new.json 2026-02-12T18:17:21.645 INFO:tasks.workunit.client.0.trial116.stdout:2 2026-02-12T18:17:21.653 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-12T18:17:21.653 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAxGY5pCAOZGBAA1tyJDMZT4eaorwEGMcYe5w== --osd-uuid 5de218c3-c7f2-4940-a6fd-2eb5d4aafaee 2026-02-12T18:17:21.671 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:21.667+0000 7f3be17fd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:21.672 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:21.669+0000 7f3be17fd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:21.673 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:21.669+0000 7f3be17fd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:21.674 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:21.670+0000 7f3be17fd780 -1 bdev(0x3a8cc00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-12T18:17:21.674 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:21.670+0000 7f3be17fd780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-12T18:17:21.991 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-12T18:17:21.991 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:17:21.992 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-12T18:17:21.992 INFO:tasks.workunit.client.0.trial116.stdout:adding osd2 key to auth repository 2026-02-12T18:17:21.992 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:17:22.318 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-12T18:17:22.319 INFO:tasks.workunit.client.0.trial116.stdout:start osd.2 2026-02-12T18:17:22.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:17:22.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:17:22.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:17:22.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:17:22.335 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:22.331+0000 7f87ec031780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:22.336 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:22.333+0000 7f87ec031780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:22.337 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:22.334+0000 7f87ec031780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:22.487 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:22.483+0000 7f87ec031780 -1 Falling back to public interface 2026-02-12T18:17:22.546 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-12T18:17:22.546 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:17:22.546 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T18:17:22.547 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:17:22.547 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:17:22.547 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:17:22.547 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:17:22.548 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:17:22.548 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:17:22.548 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:17:22.772 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:17:22.836 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:22.832+0000 7f87ec031780 -1 osd.2 0 log_to_monitors true 2026-02-12T18:17:23.773 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:17:23.774 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:17:23.774 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:17:23.775 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:17:23.775 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:17:23.775 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:17:24.002 INFO:tasks.workunit.client.0.trial116.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/4060157411,v1:127.0.0.1:6819/4060157411] [v2:127.0.0.1:6820/4060157411,v1:127.0.0.1:6821/4060157411] exists,up 5de218c3-c7f2-4940-a6fd-2eb5d4aafaee 2026-02-12T18:17:24.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:17:24.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:17:24.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:17:24.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:17:24.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-12T18:17:24.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:17:24.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:17:24.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-12T18:17:24.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:17:24.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-12T18:17:24.008 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-12T18:17:24.008 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:17:24.008 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:17:24.008 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:17:24.008 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-12T18:17:24.008 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-12T18:17:24.008 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:17:24.008 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:17:24.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:17:24.009 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:17:24.009 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:17:24.009 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:17:24.009 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:17:24.009 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:17:24.009 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:17:24.009 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:17:24.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:17:24.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:17:24.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:17:24.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:17:24.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:17:24.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:17:24.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:17:24.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:17:24.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:17:24.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:17:24.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:17:24.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:17:24.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-12T18:17:24.010 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:17:24.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=66207b8a-90c3-4ebe-b421-3c8248be4418 2026-02-12T18:17:24.011 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 66207b8a-90c3-4ebe-b421-3c8248be4418' 2026-02-12T18:17:24.011 INFO:tasks.workunit.client.0.trial116.stdout:add osd3 66207b8a-90c3-4ebe-b421-3c8248be4418 2026-02-12T18:17:24.011 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:17:24.018 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA0GY5p9jbeABAAkSVFsyB3un72CSS0g1Q+dw== 2026-02-12T18:17:24.018 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA0GY5p9jbeABAAkSVFsyB3un72CSS0g1Q+dw=="}' 2026-02-12T18:17:24.018 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 66207b8a-90c3-4ebe-b421-3c8248be4418 -i td/test-erasure-eio/3/new.json 2026-02-12T18:17:24.247 INFO:tasks.workunit.client.0.trial116.stdout:3 2026-02-12T18:17:24.255 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-12T18:17:24.256 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA0GY5p9jbeABAAkSVFsyB3un72CSS0g1Q+dw== --osd-uuid 66207b8a-90c3-4ebe-b421-3c8248be4418 2026-02-12T18:17:24.274 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:24.270+0000 7f722a631780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:24.276 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:24.272+0000 7f722a631780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:24.276 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:24.273+0000 7f722a631780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:24.277 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:24.273+0000 7f722a631780 -1 bdev(0x275cc00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-12T18:17:24.277 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:24.273+0000 7f722a631780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-12T18:17:24.605 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-12T18:17:24.606 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:17:24.606 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-12T18:17:24.607 INFO:tasks.workunit.client.0.trial116.stdout:adding osd3 key to auth repository 2026-02-12T18:17:24.607 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:17:24.932 INFO:tasks.workunit.client.0.trial116.stdout:start osd.3 2026-02-12T18:17:24.933 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-12T18:17:24.933 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:17:24.933 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:17:24.933 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:17:24.933 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:17:24.950 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:24.945+0000 7fa2479d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:24.952 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:24.946+0000 7fa2479d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:24.952 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:24.947+0000 7fa2479d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:25.096 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:25.093+0000 7fa2479d8780 -1 Falling back to public interface 2026-02-12T18:17:25.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-12T18:17:25.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:17:25.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-12T18:17:25.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:17:25.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:17:25.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:17:25.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:17:25.161 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:17:25.161 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:17:25.161 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:17:25.398 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:17:25.406 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:25.402+0000 7fa2479d8780 -1 osd.3 0 log_to_monitors true 2026-02-12T18:17:26.400 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:17:26.400 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:17:26.400 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:17:26.401 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:17:26.401 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:17:26.401 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:17:26.628 INFO:tasks.workunit.client.0.trial116.stdout:osd.3 up in weight 1 up_from 27 up_thru 28 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3640940260,v1:127.0.0.1:6827/3640940260] [v2:127.0.0.1:6828/3640940260,v1:127.0.0.1:6829/3640940260] exists,up 66207b8a-90c3-4ebe-b421-3c8248be4418 2026-02-12T18:17:26.633 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:17:26.633 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:17:26.633 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:17:26.633 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:17:26.633 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-12T18:17:26.633 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:17:26.633 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:17:26.633 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-12T18:17:26.634 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:17:26.634 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-12T18:17:26.634 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-12T18:17:26.634 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:17:26.634 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:17:26.634 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:17:26.634 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-12T18:17:26.634 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-12T18:17:26.634 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:17:26.634 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:17:26.634 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:17:26.635 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:17:26.635 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:17:26.635 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:17:26.635 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:17:26.635 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:17:26.635 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:17:26.635 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:17:26.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:17:26.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:17:26.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:17:26.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:17:26.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:17:26.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:17:26.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:17:26.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:17:26.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:17:26.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:17:26.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:17:26.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:17:26.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-12T18:17:26.636 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:17:26.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=688fea2a-43dc-4e34-abc0-276372bff079 2026-02-12T18:17:26.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 688fea2a-43dc-4e34-abc0-276372bff079' 2026-02-12T18:17:26.637 INFO:tasks.workunit.client.0.trial116.stdout:add osd4 688fea2a-43dc-4e34-abc0-276372bff079 2026-02-12T18:17:26.637 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:17:26.644 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA2GY5p/wE6JhAAZRXSuXBnHd2W2vMBdzLSbw== 2026-02-12T18:17:26.644 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA2GY5p/wE6JhAAZRXSuXBnHd2W2vMBdzLSbw=="}' 2026-02-12T18:17:26.645 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 688fea2a-43dc-4e34-abc0-276372bff079 -i td/test-erasure-eio/4/new.json 2026-02-12T18:17:26.882 INFO:tasks.workunit.client.0.trial116.stdout:4 2026-02-12T18:17:26.890 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-12T18:17:26.891 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA2GY5p/wE6JhAAZRXSuXBnHd2W2vMBdzLSbw== --osd-uuid 688fea2a-43dc-4e34-abc0-276372bff079 2026-02-12T18:17:26.908 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:26.905+0000 7f4cdb269780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:26.910 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:26.906+0000 7f4cdb269780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:26.911 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:26.907+0000 7f4cdb269780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:26.911 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:26.907+0000 7f4cdb269780 -1 bdev(0x4062c00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-12T18:17:26.911 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:26.907+0000 7f4cdb269780 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-12T18:17:27.237 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-12T18:17:27.237 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:17:27.238 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-12T18:17:27.238 INFO:tasks.workunit.client.0.trial116.stdout:adding osd4 key to auth repository 2026-02-12T18:17:27.239 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:17:27.559 INFO:tasks.workunit.client.0.trial116.stdout:start osd.4 2026-02-12T18:17:27.560 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-12T18:17:27.560 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:17:27.561 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:17:27.561 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:17:27.561 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:17:27.576 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:27.572+0000 7f3fe042d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:27.577 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:27.574+0000 7f3fe042d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:27.578 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:27.574+0000 7f3fe042d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:27.726 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:27.723+0000 7f3fe042d780 -1 Falling back to public interface 2026-02-12T18:17:27.788 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-12T18:17:27.788 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:17:27.788 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-12T18:17:27.788 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:17:27.788 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:17:27.788 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:17:27.788 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:17:27.789 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:17:27.789 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:17:27.789 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-12T18:17:28.022 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:17:28.042 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:28.038+0000 7f3fe042d780 -1 osd.4 0 log_to_monitors true 2026-02-12T18:17:29.023 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:17:29.023 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:17:29.024 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:17:29.024 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:17:29.024 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:17:29.024 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-12T18:17:29.260 INFO:tasks.workunit.client.0.trial116.stdout:osd.4 up in weight 1 up_from 36 up_thru 37 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/4118342654,v1:127.0.0.1:6835/4118342654] [v2:127.0.0.1:6836/4118342654,v1:127.0.0.1:6837/4118342654] exists,up 688fea2a-43dc-4e34-abc0-276372bff079 2026-02-12T18:17:29.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:17:29.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:17:29.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:17:29.262 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-12T18:17:29.262 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T18:17:29.262 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T18:17:29.263 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:17:29.263 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:17:29.263 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:17:29.263 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.0.asok 2026-02-12T18:17:29.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-12T18:17:29.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-osd.0.asok log flush 2026-02-12T18:17:29.306 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:17:29.309 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-12T18:17:29.312 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:17:17.584+0000 7fe0c0a23780 0 load: jerasure load: lrc 2026-02-12T18:17:29.313 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:540: TEST_ec_backfill_unfound: CEPH_ARGS='--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:17:29.313 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:542: TEST_ec_backfill_unfound: local poolname=pool-jerasure 2026-02-12T18:17:29.313 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:543: TEST_ec_backfill_unfound: create_erasure_coded_pool pool-jerasure 3 2 2026-02-12T18:17:29.313 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-12T18:17:29.313 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-12T18:17:29.313 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-12T18:17:29.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-12T18:17:29.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-12T18:17:29.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-12T18:17:29.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-12T18:17:29.599 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-12T18:17:29.599 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-12T18:17:29.975 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-jerasure' already exists 2026-02-12T18:17:29.983 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:17:30.984 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-12T18:17:30.985 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:17:30.985 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:17:30.985 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:17:30.986 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:17:30.986 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:17:30.986 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:17:30.986 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:17:30.987 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:17:30.987 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:17:31.067 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:17:31.067 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:17:31.067 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:17:31.068 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:17:31.068 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:17:31.068 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:17:31.290 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:17:31.290 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:17:31.291 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:17:31.291 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:17:31.291 INFO:tasks.workunit.client.0.trial116.stderr:4' 2026-02-12T18:17:31.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:17:31.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:17:31.291 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:17:31.354 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803781 2026-02-12T18:17:31.354 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803781 2026-02-12T18:17:31.354 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781' 2026-02-12T18:17:31.354 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:17:31.354 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:17:31.421 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574853 2026-02-12T18:17:31.421 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574853 2026-02-12T18:17:31.421 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-55834574853' 2026-02-12T18:17:31.421 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:17:31.421 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:17:31.486 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411332 2026-02-12T18:17:31.486 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411332 2026-02-12T18:17:31.486 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-55834574853 2-77309411332' 2026-02-12T18:17:31.486 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:17:31.486 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:17:31.552 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116996 2026-02-12T18:17:31.552 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116996 2026-02-12T18:17:31.552 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-55834574853 2-77309411332 3-115964116996' 2026-02-12T18:17:31.552 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:17:31.552 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-12T18:17:31.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=154618822659 2026-02-12T18:17:31.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 154618822659 2026-02-12T18:17:31.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-55834574853 2-77309411332 3-115964116996 4-154618822659' 2026-02-12T18:17:31.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:17:31.620 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803781 2026-02-12T18:17:31.620 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:17:31.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:17:31.621 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803781 2026-02-12T18:17:31.621 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:17:31.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803781 2026-02-12T18:17:31.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-12T18:17:31.622 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.0 seq 25769803781 2026-02-12T18:17:31.623 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:17:31.853 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803781 2026-02-12T18:17:31.853 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:17:31.853 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574853 2026-02-12T18:17:31.853 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:17:31.854 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:17:31.855 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574853 2026-02-12T18:17:31.855 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:17:31.856 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574853 2026-02-12T18:17:31.856 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574853' 2026-02-12T18:17:31.856 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.1 seq 55834574853 2026-02-12T18:17:31.857 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:17:32.084 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574853 -lt 55834574853 2026-02-12T18:17:32.084 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:17:32.085 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411332 2026-02-12T18:17:32.085 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:17:32.085 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:17:32.086 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411332 2026-02-12T18:17:32.086 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:17:32.087 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411332 2026-02-12T18:17:32.088 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.2 seq 77309411332 2026-02-12T18:17:32.088 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411332' 2026-02-12T18:17:32.088 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:17:32.311 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411331 -lt 77309411332 2026-02-12T18:17:32.311 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:17:33.312 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:17:33.313 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:17:33.538 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411332 -lt 77309411332 2026-02-12T18:17:33.538 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:17:33.538 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-115964116996 2026-02-12T18:17:33.538 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:17:33.539 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:17:33.540 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964116996 2026-02-12T18:17:33.540 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:17:33.541 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964116996 2026-02-12T18:17:33.541 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.3 seq 115964116996 2026-02-12T18:17:33.542 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 115964116996' 2026-02-12T18:17:33.542 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:17:33.770 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964116997 -lt 115964116996 2026-02-12T18:17:33.770 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:17:33.771 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-154618822659 2026-02-12T18:17:33.771 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:17:33.772 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-12T18:17:33.772 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-154618822659 2026-02-12T18:17:33.772 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:17:33.773 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=154618822659 2026-02-12T18:17:33.774 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 154618822659' 2026-02-12T18:17:33.774 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.4 seq 154618822659 2026-02-12T18:17:33.774 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-12T18:17:33.998 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 154618822659 -lt 154618822659 2026-02-12T18:17:33.998 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:17:33.999 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:17:33.999 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:17:34.312 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:17:34.312 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:17:34.313 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:17:34.314 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:17:34.314 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:17:34.314 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:17:34.314 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:17:34.314 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:17:34.540 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:17:34.541 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:17:34.541 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:17:34.541 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:17:34.856 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:17:34.856 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:17:34.856 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:17:34.856 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:545: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-12T18:17:35.072 INFO:tasks.workunit.client.0.trial116.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-12T18:17:35.072 INFO:tasks.workunit.client.0.trial116.stdout:2.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-12T18:17:30.506933+0000 0'0 43:16 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-12T18:17:29.908380+0000 0'0 2026-02-12T18:17:29.908380+0000 0 0 periodic scrub scheduled @ 2026-02-14T00:01:11.953625+0000 0 0 2026-02-12T18:17:35.072 INFO:tasks.workunit.client.0.trial116.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-12T18:17:29.534133+0000 0'0 43:22 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-12T18:17:15.579521+0000 0'0 2026-02-12T18:17:15.579521+0000 0 0 periodic scrub scheduled @ 2026-02-13T19:44:54.879039+0000 0 0 2026-02-12T18:17:35.072 INFO:tasks.workunit.client.0.trial116.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-12T18:17:29.534058+0000 0'0 43:99 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-12T18:17:15.579521+0000 0'0 2026-02-12T18:17:15.579521+0000 0 0 periodic scrub scheduled @ 2026-02-14T00:07:47.208061+0000 0 0 2026-02-12T18:17:35.072 INFO:tasks.workunit.client.0.trial116.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-12T18:17:29.432913+0000 0'0 43:22 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-12T18:17:15.579521+0000 0'0 2026-02-12T18:17:15.579521+0000 0 0 periodic scrub scheduled @ 2026-02-14T04:30:52.970766+0000 0 0 2026-02-12T18:17:35.073 INFO:tasks.workunit.client.0.trial116.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-12T18:17:24.014245+0000 0'0 43:66 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-12T18:17:15.579521+0000 0'0 2026-02-12T18:17:15.579521+0000 0 0 periodic scrub scheduled @ 2026-02-14T04:59:02.265426+0000 0 0 2026-02-12T18:17:35.073 INFO:tasks.workunit.client.0.trial116.stdout: 2026-02-12T18:17:35.073 INFO:tasks.workunit.client.0.trial116.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-12T18:17:35.073 INFO:tasks.workunit.client.0.trial116.stderr:dumped pgs 2026-02-12T18:17:35.080 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:547: TEST_ec_backfill_unfound: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-12T18:17:35.080 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-12T18:17:35.080 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-12T18:17:35.080 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-12T18:17:35.080 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:17:35.080 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-12T18:17:35.081 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:17:35.081 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-12T18:17:35.081 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:17:35.081 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-12T18:17:35.081 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:17:35.081 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-12T18:17:35.081 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:35.099 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:548: TEST_ec_backfill_unfound: get_primary pool-jerasure myobject 2026-02-12T18:17:35.099 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=pool-jerasure 2026-02-12T18:17:35.099 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=myobject 2026-02-12T18:17:35.099 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map pool-jerasure myobject 2026-02-12T18:17:35.099 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-12T18:17:35.334 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:548: TEST_ec_backfill_unfound: local primary=3 2026-02-12T18:17:35.335 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:550: TEST_ec_backfill_unfound: get_osds pool-jerasure myobject 2026-02-12T18:17:35.335 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:17:35.335 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-12T18:17:35.336 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-12T18:17:35.336 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:17:35.564 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:17:35.564 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:17:35.564 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:17:35.564 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:17:35.564 INFO:tasks.workunit.client.0.trial116.stderr:2' 2026-02-12T18:17:35.564 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2 2026-02-12T18:17:35.564 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:550: TEST_ec_backfill_unfound: initial_osds=('3' '1' '4' '0' '2') 2026-02-12T18:17:35.564 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:550: TEST_ec_backfill_unfound: local -a initial_osds 2026-02-12T18:17:35.564 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:551: TEST_ec_backfill_unfound: local last_osd=2 2026-02-12T18:17:35.564 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:552: TEST_ec_backfill_unfound: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-12T18:17:35.565 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:17:35.565 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:17:35.565 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:17:35.565 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:17:35.565 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:17:35.670 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:17:35.670 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:553: TEST_ec_backfill_unfound: ceph osd down 2 2026-02-12T18:17:35.900 INFO:tasks.workunit.client.0.trial116.stderr:osd.2 is already down. 2026-02-12T18:17:35.908 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:554: TEST_ec_backfill_unfound: ceph osd out 2 2026-02-12T18:17:36.193 INFO:tasks.workunit.client.0.trial116.stderr:osd.2 is already out. 2026-02-12T18:17:36.202 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:556: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-12T18:17:36.417 INFO:tasks.workunit.client.0.trial116.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-12T18:17:36.418 INFO:tasks.workunit.client.0.trial116.stdout:2.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-12T18:17:30.506933+0000 0'0 43:16 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-12T18:17:29.908380+0000 0'0 2026-02-12T18:17:29.908380+0000 0 0 periodic scrub scheduled @ 2026-02-14T00:01:11.953625+0000 0 0 2026-02-12T18:17:36.418 INFO:tasks.workunit.client.0.trial116.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-12T18:17:29.534133+0000 0'0 43:22 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-12T18:17:15.579521+0000 0'0 2026-02-12T18:17:15.579521+0000 0 0 periodic scrub scheduled @ 2026-02-13T19:44:54.879039+0000 0 0 2026-02-12T18:17:36.418 INFO:tasks.workunit.client.0.trial116.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-12T18:17:29.534058+0000 0'0 43:99 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-12T18:17:15.579521+0000 0'0 2026-02-12T18:17:15.579521+0000 0 0 periodic scrub scheduled @ 2026-02-14T00:07:47.208061+0000 0 0 2026-02-12T18:17:36.418 INFO:tasks.workunit.client.0.trial116.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-12T18:17:29.432913+0000 0'0 43:22 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-12T18:17:15.579521+0000 0'0 2026-02-12T18:17:15.579521+0000 0 0 periodic scrub scheduled @ 2026-02-14T04:30:52.970766+0000 0 0 2026-02-12T18:17:36.418 INFO:tasks.workunit.client.0.trial116.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-12T18:17:24.014245+0000 0'0 43:66 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-12T18:17:15.579521+0000 0'0 2026-02-12T18:17:15.579521+0000 0 0 periodic scrub scheduled @ 2026-02-14T04:59:02.265426+0000 0 0 2026-02-12T18:17:36.418 INFO:tasks.workunit.client.0.trial116.stdout: 2026-02-12T18:17:36.418 INFO:tasks.workunit.client.0.trial116.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-12T18:17:36.419 INFO:tasks.workunit.client.0.trial116.stderr:dumped pgs 2026-02-12T18:17:36.424 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:558: TEST_ec_backfill_unfound: dd if=/dev/urandom of=td/test-erasure-eio/ORIGINAL bs=1024 count=4 2026-02-12T18:17:36.425 INFO:tasks.workunit.client.0.trial116.stderr:4+0 records in 2026-02-12T18:17:36.425 INFO:tasks.workunit.client.0.trial116.stderr:4+0 records out 2026-02-12T18:17:36.425 INFO:tasks.workunit.client.0.trial116.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 6.7535e-05 s, 60.7 MB/s 2026-02-12T18:17:36.426 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: seq 1 300 2026-02-12T18:17:36.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj1 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.445 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.445 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj2 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.461 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.461 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj3 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.478 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.478 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj4 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.494 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.494 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj5 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.512 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.512 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj6 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.528 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.529 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj7 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.545 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.545 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj8 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.563 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.564 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj9 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.581 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.581 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj10 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.598 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.598 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj11 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.614 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.615 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj12 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.632 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.632 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj13 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.649 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.649 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj14 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.667 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.667 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj15 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.684 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.684 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj16 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.700 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.700 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj17 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.717 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.717 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj18 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.734 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.734 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj19 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.750 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.750 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj20 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.767 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.767 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj21 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj22 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.800 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.800 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj23 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.816 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.816 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj24 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.833 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.833 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj25 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.849 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.850 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj26 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.866 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.866 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj27 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.882 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj28 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.900 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.900 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj29 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.917 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.917 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj30 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.933 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.933 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj31 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.950 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.950 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj32 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.966 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.966 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj33 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.982 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.983 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj34 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:36.999 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:36.999 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj35 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.016 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.016 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj36 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.033 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.033 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj37 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.050 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.050 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj38 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.066 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.066 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj39 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.083 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.084 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj40 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.100 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.101 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj41 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.117 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.117 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj42 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj43 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.152 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.152 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj44 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.168 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.168 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj45 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.184 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.184 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj46 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.201 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.201 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj47 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.218 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.218 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj48 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.235 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.235 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj49 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.252 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.252 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj50 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.269 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.269 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj51 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.286 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.286 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj52 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.302 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.302 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj53 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.319 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.319 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj54 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.336 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.336 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj55 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.352 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.353 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj56 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.370 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.370 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj57 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.386 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.386 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj58 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.403 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.403 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj59 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.420 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.420 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj60 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.437 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.438 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj61 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.454 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.454 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj62 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.471 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.471 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj63 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.488 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.488 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj64 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.504 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.505 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj65 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.522 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.522 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj66 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.539 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.539 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj67 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.556 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.556 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj68 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.573 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.573 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj69 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.590 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj70 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.607 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.607 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj71 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.623 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.623 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj72 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj73 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.657 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.657 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj74 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.673 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj75 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.690 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.691 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj76 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.707 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.707 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj77 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.724 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.724 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj78 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.741 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.741 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj79 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.757 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.757 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj80 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.775 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.775 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj81 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.792 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.792 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj82 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj83 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.825 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.826 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj84 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.843 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.843 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj85 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.860 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.860 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj86 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.877 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.877 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj87 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.893 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.893 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj88 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.911 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.911 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj89 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.927 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.927 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj90 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.944 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.944 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj91 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.960 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.961 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj92 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.977 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.977 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj93 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:37.994 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:37.994 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj94 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj95 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj96 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj97 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj98 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.078 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.078 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj99 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.095 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.095 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj100 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.112 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.112 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj101 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.128 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.128 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj102 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.145 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.145 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj103 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.161 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.161 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj104 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.178 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.178 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj105 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.194 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.194 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj106 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.211 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.211 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj107 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.228 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.228 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj108 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj109 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj110 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.278 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.278 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj111 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.295 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.295 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj112 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.311 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.311 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj113 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj114 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.345 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.345 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj115 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.361 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.362 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj116 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.378 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.378 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj117 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.395 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.395 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj118 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.412 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.412 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj119 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.429 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.429 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj120 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.445 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.445 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj121 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.461 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.461 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj122 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.478 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.478 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj123 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.494 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.495 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj124 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.512 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.512 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj125 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.528 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.528 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj126 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.545 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.545 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj127 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.561 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.561 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj128 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.577 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.577 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj129 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.594 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.594 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj130 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.611 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.611 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj131 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj132 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.643 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.643 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj133 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.660 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.660 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj134 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.676 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.676 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj135 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.693 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.693 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj136 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.710 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.710 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj137 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.726 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.727 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj138 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.743 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.743 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj139 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.760 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.760 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj140 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.776 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.777 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj141 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.793 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.793 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj142 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.810 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.810 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj143 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.826 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.826 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj144 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.842 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.842 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj145 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj146 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.875 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.875 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj147 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.891 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.891 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj148 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.908 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.908 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj149 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.924 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.925 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj150 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.941 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.942 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj151 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.958 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.958 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj152 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.976 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.976 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj153 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:38.992 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:38.992 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj154 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj155 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.025 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.025 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj156 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.041 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.041 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj157 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.058 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.058 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj158 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.074 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.075 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj159 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.091 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.092 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj160 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.109 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.109 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj161 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.126 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.126 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj162 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj163 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.159 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj164 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.177 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.177 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj165 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.193 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.193 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj166 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.210 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.211 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj167 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.227 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.227 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj168 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj169 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj170 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.278 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.278 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj171 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.295 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.295 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj172 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.312 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.312 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj173 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.330 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.330 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj174 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.347 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.347 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj175 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.363 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.363 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj176 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.380 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.380 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj177 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.397 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.397 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj178 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.413 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.413 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj179 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj180 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.447 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.447 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj181 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.463 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.463 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj182 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.480 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.480 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj183 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.497 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.498 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj184 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.514 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.514 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj185 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.532 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.532 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj186 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.549 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.549 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj187 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.567 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.567 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj188 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.583 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.583 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj189 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.599 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.599 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj190 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.615 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.615 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj191 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.632 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.632 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj192 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.649 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.649 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj193 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.665 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.665 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj194 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.681 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.681 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj195 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.698 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.698 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj196 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.715 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.715 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj197 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj198 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj199 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.764 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.765 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj200 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.781 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.781 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj201 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.798 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.798 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj202 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.814 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.814 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj203 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.831 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.831 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj204 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.849 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.849 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj205 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.866 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.866 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj206 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj207 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.899 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.899 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj208 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.916 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.916 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj209 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.933 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.933 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj210 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.950 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.950 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj211 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.967 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.967 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj212 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:39.983 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:39.984 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj213 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.001 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.001 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj214 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.017 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.017 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj215 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.033 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.034 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj216 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.050 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.050 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj217 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.067 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.067 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj218 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.084 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.084 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj219 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.101 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.101 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj220 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.117 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.118 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj221 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj222 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.151 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.151 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj223 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.168 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.168 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj224 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.184 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.184 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj225 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.201 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.201 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj226 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj227 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.235 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj228 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.250 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.251 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj229 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.267 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.267 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj230 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.284 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.284 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj231 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.301 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.301 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj232 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.317 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.317 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj233 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.333 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.334 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj234 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.350 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.350 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj235 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.367 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.367 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj236 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.383 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.383 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj237 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.400 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.400 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj238 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj239 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj240 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.449 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.449 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj241 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.466 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.466 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj242 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.484 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.484 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj243 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.500 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.500 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj244 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.517 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.517 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj245 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.534 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.534 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj246 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.550 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.550 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj247 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.568 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.568 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj248 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.584 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.584 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj249 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.601 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.601 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj250 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.618 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.618 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj251 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj252 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.653 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.653 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj253 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.669 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.670 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj254 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj255 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.702 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.702 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj256 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj257 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.736 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.736 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj258 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.752 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.752 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj259 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj260 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.785 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.786 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj261 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj262 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.818 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.818 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj263 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.835 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.835 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj264 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.852 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.852 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj265 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.869 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.869 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj266 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.885 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.885 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj267 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj268 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.919 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.919 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj269 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.935 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.935 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj270 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.952 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.952 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj271 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.969 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.969 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj272 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:40.985 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:40.985 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj273 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:41.001 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:41.001 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj274 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:41.017 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:41.017 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj275 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:41.034 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:41.034 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj276 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:41.051 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:41.051 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj277 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:41.067 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:41.068 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj278 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:41.085 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:41.085 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj279 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:41.101 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:41.101 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj280 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:41.118 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:41.118 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj281 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:41.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:41.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj282 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:41.152 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:41.152 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj283 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:41.169 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:41.169 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj284 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:41.185 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:41.185 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj285 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:41.201 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:41.201 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj286 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:41.218 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:41.218 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj287 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:41.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:41.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj288 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:41.250 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:41.251 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj289 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:41.267 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:41.268 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj290 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:41.283 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:41.284 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj291 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:41.300 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:41.300 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj292 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:41.317 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:41.317 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj293 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:41.333 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:41.334 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj294 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:41.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:41.350 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj295 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:41.366 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:41.366 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj296 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:41.382 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:41.382 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj297 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:41.400 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:41.400 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj298 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:41.416 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:41.416 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj299 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:41.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:17:41.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj300 td/test-erasure-eio/ORIGINAL 2026-02-12T18:17:41.451 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:564: TEST_ec_backfill_unfound: inject_eio ec data pool-jerasure obj250 td/test-erasure-eio 0 2026-02-12T18:17:41.451 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-12T18:17:41.451 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-12T18:17:41.451 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-12T18:17:41.451 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-12T18:17:41.451 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-12T18:17:41.451 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-12T18:17:41.451 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj250 2026-02-12T18:17:41.452 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-12T18:17:41.452 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-12T18:17:41.452 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-12T18:17:41.452 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=0 2026-02-12T18:17:41.452 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-12T18:17:41.452 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj250 2026-02-12T18:17:41.452 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:17:41.452 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj250 2026-02-12T18:17:41.452 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj250 2026-02-12T18:17:41.452 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:17:41.684 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:17:41.685 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:17:41.685 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:17:41.685 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:17:41.685 INFO:tasks.workunit.client.0.trial116.stderr:2147483647' 2026-02-12T18:17:41.685 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-12T18:17:41.685 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-12T18:17:41.685 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-12T18:17:41.685 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=3 2026-02-12T18:17:41.685 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-12T18:17:41.685 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/3/type 2026-02-12T18:17:41.688 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-12T18:17:41.688 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-12T18:17:41.688 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-12T18:17:41.688 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-12T18:17:41.688 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-12T18:17:41.688 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-12T18:17:41.688 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-12T18:17:41.689 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-12T18:17:41.689 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-12T18:17:41.689 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-12T18:17:41.689 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:17:41.689 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:17:41.689 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:17:41.689 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.3.asok 2026-02-12T18:17:41.689 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.76821/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-12T18:17:41.734 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-12T18:17:41.734 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-12T18:17:41.735 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-12T18:17:41.735 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.3 2026-02-12T18:17:41.735 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-12T18:17:41.735 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-12T18:17:41.735 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:17:41.735 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:17:41.735 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:17:41.736 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.3.asok 2026-02-12T18:17:41.736 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-12T18:17:41.736 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-osd.3.asok injectdataerr pool-jerasure obj250 0 2026-02-12T18:17:41.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:565: TEST_ec_backfill_unfound: inject_eio ec data pool-jerasure obj250 td/test-erasure-eio 1 2026-02-12T18:17:41.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-12T18:17:41.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-12T18:17:41.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-12T18:17:41.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-12T18:17:41.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-12T18:17:41.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-12T18:17:41.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj250 2026-02-12T18:17:41.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-12T18:17:41.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-12T18:17:41.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-12T18:17:41.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=1 2026-02-12T18:17:41.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-12T18:17:41.783 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj250 2026-02-12T18:17:41.784 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:17:41.784 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj250 2026-02-12T18:17:41.784 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj250 2026-02-12T18:17:41.784 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:17:42.011 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:17:42.011 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:17:42.012 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:17:42.012 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:17:42.012 INFO:tasks.workunit.client.0.trial116.stderr:2147483647' 2026-02-12T18:17:42.012 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-12T18:17:42.012 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-12T18:17:42.012 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-12T18:17:42.012 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=1 2026-02-12T18:17:42.012 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-12T18:17:42.012 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/1/type 2026-02-12T18:17:42.014 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-12T18:17:42.014 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-12T18:17:42.015 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-12T18:17:42.015 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-12T18:17:42.015 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-12T18:17:42.015 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-12T18:17:42.015 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-12T18:17:42.015 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-12T18:17:42.015 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-12T18:17:42.015 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-12T18:17:42.015 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:17:42.015 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:17:42.015 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:17:42.015 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.1.asok 2026-02-12T18:17:42.016 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.76821/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-12T18:17:42.063 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-12T18:17:42.063 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-12T18:17:42.064 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-12T18:17:42.064 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.1 2026-02-12T18:17:42.064 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-12T18:17:42.064 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-12T18:17:42.064 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:17:42.064 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:17:42.064 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:17:42.065 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.1.asok 2026-02-12T18:17:42.065 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-12T18:17:42.065 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-osd.1.asok injectdataerr pool-jerasure obj250 1 2026-02-12T18:17:42.112 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:567: TEST_ec_backfill_unfound: activate_osd td/test-erasure-eio 2 2026-02-12T18:17:42.112 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-12T18:17:42.112 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T18:17:42.113 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-12T18:17:42.113 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T18:17:42.113 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-12T18:17:42.113 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:17:42.113 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:17:42.113 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:17:42.113 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:17:42.113 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-12T18:17:42.113 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-12T18:17:42.113 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T18:17:42.113 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T18:17:42.113 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:17:42.113 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T18:17:42.113 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:17:42.114 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:17:42.115 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:17:42.116 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:17:42.116 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:17:42.116 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:17:42.116 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:17:42.116 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:17:42.116 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:17:42.116 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:17:42.116 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:17:42.116 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:17:42.116 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:17:42.116 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:17:42.116 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T18:17:42.117 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-12T18:17:42.117 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-12T18:17:42.117 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-12T18:17:42.117 INFO:tasks.workunit.client.0.trial116.stdout:start osd.2 2026-02-12T18:17:42.118 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:17:42.118 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-12T18:17:42.118 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-12T18:17:42.118 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T18:17:42.118 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T18:17:42.118 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T18:17:42.130 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:42.127+0000 7f8f16632780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:42.131 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:42.129+0000 7f8f16632780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:42.132 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:42.130+0000 7f8f16632780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:17:42.259 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:42.256+0000 7f8f16632780 -1 Falling back to public interface 2026-02-12T18:17:42.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-12T18:17:42.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:17:42.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T18:17:42.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:17:42.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:17:42.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:17:42.347 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:17:42.347 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:17:42.347 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:17:42.347 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:17:42.562 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:17:42.559+0000 7f8f16632780 -1 osd.2 43 log_to_monitors true 2026-02-12T18:17:42.575 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:17:43.577 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:17:43.577 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:17:43.577 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:17:43.578 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:17:43.578 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:17:43.578 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:17:43.806 INFO:tasks.workunit.client.0.trial116.stdout:osd.2 up out weight 0 up_from 49 up_thru 20 down_at 44 last_clean_interval [18,43) [v2:127.0.0.1:6818/2725303290,v1:127.0.0.1:6819/2725303290] [v2:127.0.0.1:6820/2725303290,v1:127.0.0.1:6821/2725303290] exists,up 5de218c3-c7f2-4940-a6fd-2eb5d4aafaee 2026-02-12T18:17:43.807 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:17:43.807 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:17:43.807 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:17:43.807 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:568: TEST_ec_backfill_unfound: ceph osd in 2 2026-02-12T18:17:44.037 INFO:tasks.workunit.client.0.trial116.stderr:osd.2 is already in. 2026-02-12T18:17:44.045 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:570: TEST_ec_backfill_unfound: sleep 15 2026-02-12T18:17:59.047 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:572: TEST_ec_backfill_unfound: seq 1 240 2026-02-12T18:17:59.048 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:572: TEST_ec_backfill_unfound: for tmp in $(seq 1 240) 2026-02-12T18:17:59.049 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:573: TEST_ec_backfill_unfound: get_state 2.0 2026-02-12T18:17:59.049 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:60: get_state: local pgid=2.0 2026-02-12T18:17:59.049 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:61: get_state: local sname=state 2026-02-12T18:17:59.050 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:62: get_state: ceph --format json pg dump pgs 2026-02-12T18:17:59.050 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:63: get_state: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .state' 2026-02-12T18:17:59.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:573: TEST_ec_backfill_unfound: state=active+backfill_unfound+undersized+degraded+remapped 2026-02-12T18:17:59.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:574: TEST_ec_backfill_unfound: echo active+backfill_unfound+undersized+degraded+remapped 2026-02-12T18:17:59.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:574: TEST_ec_backfill_unfound: grep backfill_unfound 2026-02-12T18:17:59.276 INFO:tasks.workunit.client.0.trial116.stdout:active+backfill_unfound+undersized+degraded+remapped 2026-02-12T18:17:59.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:575: TEST_ec_backfill_unfound: '[' 0 = 0 ']' 2026-02-12T18:17:59.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:576: TEST_ec_backfill_unfound: break 2026-02-12T18:17:59.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:582: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-12T18:17:59.497 INFO:tasks.workunit.client.0.trial116.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-12T18:17:59.497 INFO:tasks.workunit.client.0.trial116.stdout:2.0 301 1 28 0 1 1232896 0 0 101 200 101 active+backfill_unfound+undersized+degraded+remapped 2026-02-12T18:17:44.774991+0000 47'301 55:907 [3,1,4,0,2] 3 [3,1,4,0,NONE] 3 0'0 2026-02-12T18:17:29.908380+0000 0'0 2026-02-12T18:17:29.908380+0000 0 0 no scrub is scheduled 0 0 2026-02-12T18:17:59.497 INFO:tasks.workunit.client.0.trial116.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-12T18:17:44.070982+0000 0'0 55:63 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-12T18:17:15.579521+0000 0'0 2026-02-12T18:17:15.579521+0000 0 0 periodic scrub scheduled @ 2026-02-14T00:15:29.057819+0000 0 0 2026-02-12T18:17:59.498 INFO:tasks.workunit.client.0.trial116.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-12T18:17:29.534058+0000 0'0 55:122 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-12T18:17:15.579521+0000 0'0 2026-02-12T18:17:15.579521+0000 0 0 periodic scrub scheduled @ 2026-02-14T00:07:47.208061+0000 0 0 2026-02-12T18:17:59.498 INFO:tasks.workunit.client.0.trial116.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-12T18:17:29.432913+0000 0'0 55:45 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-12T18:17:15.579521+0000 0'0 2026-02-12T18:17:15.579521+0000 0 0 periodic scrub scheduled @ 2026-02-14T04:30:52.970766+0000 0 0 2026-02-12T18:17:59.498 INFO:tasks.workunit.client.0.trial116.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-12T18:17:44.071072+0000 0'0 55:108 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-12T18:17:15.579521+0000 0'0 2026-02-12T18:17:15.579521+0000 0 0 periodic scrub scheduled @ 2026-02-14T04:12:19.680304+0000 0 0 2026-02-12T18:17:59.498 INFO:tasks.workunit.client.0.trial116.stdout: 2026-02-12T18:17:59.498 INFO:tasks.workunit.client.0.trial116.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-12T18:17:59.498 INFO:tasks.workunit.client.0.trial116.stderr:dumped pgs 2026-02-12T18:17:59.505 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:583: TEST_ec_backfill_unfound: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-12T18:17:59.506 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:17:59.506 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:17:59.506 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:17:59.506 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:17:59.506 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:17:59.611 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:17:59.611 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:584: TEST_ec_backfill_unfound: sleep 5 2026-02-12T18:18:04.612 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:586: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-12T18:18:04.828 INFO:tasks.workunit.client.0.trial116.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-12T18:18:04.829 INFO:tasks.workunit.client.0.trial116.stdout:2.0 301 1 303 0 1 1232896 0 0 101 200 101 active+recovery_unfound+undersized+degraded 2026-02-12T18:17:59.663356+0000 47'301 57:919 [3,1,4,0,NONE] 3 [3,1,4,0,NONE] 3 0'0 2026-02-12T18:17:29.908380+0000 0'0 2026-02-12T18:17:29.908380+0000 0 0 no scrub is scheduled 0 0 2026-02-12T18:18:04.829 INFO:tasks.workunit.client.0.trial116.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-12T18:17:44.070982+0000 0'0 55:63 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-12T18:17:15.579521+0000 0'0 2026-02-12T18:17:15.579521+0000 0 0 periodic scrub scheduled @ 2026-02-14T00:15:29.057819+0000 0 0 2026-02-12T18:18:04.829 INFO:tasks.workunit.client.0.trial116.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-12T18:17:29.534058+0000 0'0 55:122 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-12T18:17:15.579521+0000 0'0 2026-02-12T18:17:15.579521+0000 0 0 periodic scrub scheduled @ 2026-02-14T00:07:47.208061+0000 0 0 2026-02-12T18:18:04.829 INFO:tasks.workunit.client.0.trial116.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-12T18:17:29.432913+0000 0'0 55:45 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-12T18:17:15.579521+0000 0'0 2026-02-12T18:17:15.579521+0000 0 0 periodic scrub scheduled @ 2026-02-14T04:30:52.970766+0000 0 0 2026-02-12T18:18:04.829 INFO:tasks.workunit.client.0.trial116.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+undersized 2026-02-12T18:17:59.661789+0000 0'0 57:116 [1,0] 1 [1,0] 1 0'0 2026-02-12T18:17:15.579521+0000 0'0 2026-02-12T18:17:15.579521+0000 0 0 periodic scrub scheduled @ 2026-02-13T20:52:05.055673+0000 0 0 2026-02-12T18:18:04.829 INFO:tasks.workunit.client.0.trial116.stdout: 2026-02-12T18:18:04.829 INFO:tasks.workunit.client.0.trial116.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-12T18:18:04.830 INFO:tasks.workunit.client.0.trial116.stderr:dumped pgs 2026-02-12T18:18:04.836 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:587: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-12T18:18:04.897 INFO:tasks.workunit.client.0.trial116.stdout:{ 2026-02-12T18:18:04.897 INFO:tasks.workunit.client.0.trial116.stdout: "num_missing": 1, 2026-02-12T18:18:04.897 INFO:tasks.workunit.client.0.trial116.stdout: "num_unfound": 1, 2026-02-12T18:18:04.898 INFO:tasks.workunit.client.0.trial116.stdout: "objects": [ 2026-02-12T18:18:04.898 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:18:04.898 INFO:tasks.workunit.client.0.trial116.stdout: "oid": { 2026-02-12T18:18:04.898 INFO:tasks.workunit.client.0.trial116.stdout: "oid": "obj250", 2026-02-12T18:18:04.898 INFO:tasks.workunit.client.0.trial116.stdout: "key": "", 2026-02-12T18:18:04.898 INFO:tasks.workunit.client.0.trial116.stdout: "snapid": -2, 2026-02-12T18:18:04.898 INFO:tasks.workunit.client.0.trial116.stdout: "hash": 2249616407, 2026-02-12T18:18:04.898 INFO:tasks.workunit.client.0.trial116.stdout: "max": 0, 2026-02-12T18:18:04.898 INFO:tasks.workunit.client.0.trial116.stdout: "pool": 2, 2026-02-12T18:18:04.898 INFO:tasks.workunit.client.0.trial116.stdout: "namespace": "" 2026-02-12T18:18:04.898 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:18:04.898 INFO:tasks.workunit.client.0.trial116.stdout: "need": "47'251", 2026-02-12T18:18:04.898 INFO:tasks.workunit.client.0.trial116.stdout: "have": "0'0", 2026-02-12T18:18:04.899 INFO:tasks.workunit.client.0.trial116.stdout: "flags": "none", 2026-02-12T18:18:04.899 INFO:tasks.workunit.client.0.trial116.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2026-02-12T18:18:04.899 INFO:tasks.workunit.client.0.trial116.stdout: "locations": [ 2026-02-12T18:18:04.899 INFO:tasks.workunit.client.0.trial116.stdout: "0(3)", 2026-02-12T18:18:04.899 INFO:tasks.workunit.client.0.trial116.stdout: "4(2)" 2026-02-12T18:18:04.899 INFO:tasks.workunit.client.0.trial116.stdout: ] 2026-02-12T18:18:04.899 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:18:04.899 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:18:04.899 INFO:tasks.workunit.client.0.trial116.stdout: "state": "NotRecovering", 2026-02-12T18:18:04.899 INFO:tasks.workunit.client.0.trial116.stdout: "available_might_have_unfound": true, 2026-02-12T18:18:04.899 INFO:tasks.workunit.client.0.trial116.stdout: "might_have_unfound": [ 2026-02-12T18:18:04.900 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:18:04.900 INFO:tasks.workunit.client.0.trial116.stdout: "osd": "2(4)", 2026-02-12T18:18:04.900 INFO:tasks.workunit.client.0.trial116.stdout: "status": "osd is down" 2026-02-12T18:18:04.900 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:18:04.900 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:18:04.900 INFO:tasks.workunit.client.0.trial116.stdout: "more": false 2026-02-12T18:18:04.900 INFO:tasks.workunit.client.0.trial116.stdout:} 2026-02-12T18:18:04.901 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:588: TEST_ec_backfill_unfound: ceph pg 2.0 query 2026-02-12T18:18:04.964 INFO:tasks.workunit.client.0.trial116.stdout:{ 2026-02-12T18:18:04.964 INFO:tasks.workunit.client.0.trial116.stdout: "snap_trimq": "[]", 2026-02-12T18:18:04.964 INFO:tasks.workunit.client.0.trial116.stdout: "snap_trimq_len": 0, 2026-02-12T18:18:04.964 INFO:tasks.workunit.client.0.trial116.stdout: "state": "active+recovery_unfound+undersized+degraded", 2026-02-12T18:18:04.964 INFO:tasks.workunit.client.0.trial116.stdout: "epoch": 57, 2026-02-12T18:18:04.965 INFO:tasks.workunit.client.0.trial116.stdout: "up": [ 2026-02-12T18:18:04.965 INFO:tasks.workunit.client.0.trial116.stdout: 3, 2026-02-12T18:18:04.965 INFO:tasks.workunit.client.0.trial116.stdout: 1, 2026-02-12T18:18:04.965 INFO:tasks.workunit.client.0.trial116.stdout: 4, 2026-02-12T18:18:04.965 INFO:tasks.workunit.client.0.trial116.stdout: 0, 2026-02-12T18:18:04.965 INFO:tasks.workunit.client.0.trial116.stdout: 2147483647 2026-02-12T18:18:04.965 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:18:04.965 INFO:tasks.workunit.client.0.trial116.stdout: "acting": [ 2026-02-12T18:18:04.965 INFO:tasks.workunit.client.0.trial116.stdout: 3, 2026-02-12T18:18:04.965 INFO:tasks.workunit.client.0.trial116.stdout: 1, 2026-02-12T18:18:04.965 INFO:tasks.workunit.client.0.trial116.stdout: 4, 2026-02-12T18:18:04.965 INFO:tasks.workunit.client.0.trial116.stdout: 0, 2026-02-12T18:18:04.966 INFO:tasks.workunit.client.0.trial116.stdout: 2147483647 2026-02-12T18:18:04.966 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:18:04.966 INFO:tasks.workunit.client.0.trial116.stdout: "acting_recovery_backfill": [ 2026-02-12T18:18:04.966 INFO:tasks.workunit.client.0.trial116.stdout: "0(3)", 2026-02-12T18:18:04.966 INFO:tasks.workunit.client.0.trial116.stdout: "1(1)", 2026-02-12T18:18:04.966 INFO:tasks.workunit.client.0.trial116.stdout: "3(0)", 2026-02-12T18:18:04.966 INFO:tasks.workunit.client.0.trial116.stdout: "4(2)" 2026-02-12T18:18:04.966 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:18:04.966 INFO:tasks.workunit.client.0.trial116.stdout: "info": { 2026-02-12T18:18:04.966 INFO:tasks.workunit.client.0.trial116.stdout: "pgid": "2.0s0", 2026-02-12T18:18:04.966 INFO:tasks.workunit.client.0.trial116.stdout: "shard": "0", 2026-02-12T18:18:04.966 INFO:tasks.workunit.client.0.trial116.stdout: "last_update": "47'301", 2026-02-12T18:18:04.967 INFO:tasks.workunit.client.0.trial116.stdout: "last_complete": "47'250", 2026-02-12T18:18:04.967 INFO:tasks.workunit.client.0.trial116.stdout: "log_tail": "47'200", 2026-02-12T18:18:04.967 INFO:tasks.workunit.client.0.trial116.stdout: "last_user_version": 301, 2026-02-12T18:18:04.967 INFO:tasks.workunit.client.0.trial116.stdout: "last_backfill": "MAX", 2026-02-12T18:18:04.967 INFO:tasks.workunit.client.0.trial116.stdout: "partial_writes_last_complete": [], 2026-02-12T18:18:04.967 INFO:tasks.workunit.client.0.trial116.stdout: "partial_writes_last_complete_epoch": "56", 2026-02-12T18:18:04.967 INFO:tasks.workunit.client.0.trial116.stdout: "purged_snaps": [], 2026-02-12T18:18:04.967 INFO:tasks.workunit.client.0.trial116.stdout: "history": { 2026-02-12T18:18:04.967 INFO:tasks.workunit.client.0.trial116.stdout: "epoch_created": 41, 2026-02-12T18:18:04.967 INFO:tasks.workunit.client.0.trial116.stdout: "epoch_pool_created": 41, 2026-02-12T18:18:04.967 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_started": 57, 2026-02-12T18:18:04.967 INFO:tasks.workunit.client.0.trial116.stdout: "last_interval_started": 56, 2026-02-12T18:18:04.968 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_clean": 42, 2026-02-12T18:18:04.968 INFO:tasks.workunit.client.0.trial116.stdout: "last_interval_clean": 41, 2026-02-12T18:18:04.968 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_split": 0, 2026-02-12T18:18:04.968 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_marked_full": 0, 2026-02-12T18:18:04.968 INFO:tasks.workunit.client.0.trial116.stdout: "same_up_since": 56, 2026-02-12T18:18:04.968 INFO:tasks.workunit.client.0.trial116.stdout: "same_interval_since": 56, 2026-02-12T18:18:04.968 INFO:tasks.workunit.client.0.trial116.stdout: "same_primary_since": 41, 2026-02-12T18:18:04.968 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub": "0'0", 2026-02-12T18:18:04.968 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub_stamp": "2026-02-12T18:17:29.908380+0000", 2026-02-12T18:18:04.968 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub": "0'0", 2026-02-12T18:18:04.968 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub_stamp": "2026-02-12T18:17:29.908380+0000", 2026-02-12T18:18:04.968 INFO:tasks.workunit.client.0.trial116.stdout: "last_clean_scrub_stamp": "2026-02-12T18:17:29.908380+0000", 2026-02-12T18:18:04.969 INFO:tasks.workunit.client.0.trial116.stdout: "prior_readable_until_ub": 0 2026-02-12T18:18:04.969 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:18:04.969 INFO:tasks.workunit.client.0.trial116.stdout: "stats": { 2026-02-12T18:18:04.969 INFO:tasks.workunit.client.0.trial116.stdout: "version": "47'301", 2026-02-12T18:18:04.969 INFO:tasks.workunit.client.0.trial116.stdout: "reported_seq": 920, 2026-02-12T18:18:04.969 INFO:tasks.workunit.client.0.trial116.stdout: "reported_epoch": 57, 2026-02-12T18:18:04.969 INFO:tasks.workunit.client.0.trial116.stdout: "state": "active+recovery_unfound+undersized+degraded", 2026-02-12T18:18:04.969 INFO:tasks.workunit.client.0.trial116.stdout: "last_fresh": "2026-02-12T18:17:59.663356+0000", 2026-02-12T18:18:04.969 INFO:tasks.workunit.client.0.trial116.stdout: "last_change": "2026-02-12T18:17:59.663356+0000", 2026-02-12T18:18:04.969 INFO:tasks.workunit.client.0.trial116.stdout: "last_active": "2026-02-12T18:17:59.663356+0000", 2026-02-12T18:18:04.969 INFO:tasks.workunit.client.0.trial116.stdout: "last_peered": "2026-02-12T18:17:59.663356+0000", 2026-02-12T18:18:04.969 INFO:tasks.workunit.client.0.trial116.stdout: "last_clean": "2026-02-12T18:17:35.094275+0000", 2026-02-12T18:18:04.970 INFO:tasks.workunit.client.0.trial116.stdout: "last_became_active": "2026-02-12T18:17:59.662077+0000", 2026-02-12T18:18:04.970 INFO:tasks.workunit.client.0.trial116.stdout: "last_became_peered": "2026-02-12T18:17:59.662077+0000", 2026-02-12T18:18:04.970 INFO:tasks.workunit.client.0.trial116.stdout: "last_unstale": "2026-02-12T18:17:59.663356+0000", 2026-02-12T18:18:04.970 INFO:tasks.workunit.client.0.trial116.stdout: "last_undegraded": "2026-02-12T18:17:59.660919+0000", 2026-02-12T18:18:04.970 INFO:tasks.workunit.client.0.trial116.stdout: "last_fullsized": "2026-02-12T18:17:59.660789+0000", 2026-02-12T18:18:04.970 INFO:tasks.workunit.client.0.trial116.stdout: "mapping_epoch": 56, 2026-02-12T18:18:04.970 INFO:tasks.workunit.client.0.trial116.stdout: "log_start": "47'200", 2026-02-12T18:18:04.970 INFO:tasks.workunit.client.0.trial116.stdout: "ondisk_log_start": "47'200", 2026-02-12T18:18:04.970 INFO:tasks.workunit.client.0.trial116.stdout: "created": 41, 2026-02-12T18:18:04.970 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_clean": 42, 2026-02-12T18:18:04.970 INFO:tasks.workunit.client.0.trial116.stdout: "parent": "0.0", 2026-02-12T18:18:04.970 INFO:tasks.workunit.client.0.trial116.stdout: "parent_split_bits": 0, 2026-02-12T18:18:04.970 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub": "0'0", 2026-02-12T18:18:04.971 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub_stamp": "2026-02-12T18:17:29.908380+0000", 2026-02-12T18:18:04.971 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub": "0'0", 2026-02-12T18:18:04.971 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub_stamp": "2026-02-12T18:17:29.908380+0000", 2026-02-12T18:18:04.971 INFO:tasks.workunit.client.0.trial116.stdout: "last_clean_scrub_stamp": "2026-02-12T18:17:29.908380+0000", 2026-02-12T18:18:04.971 INFO:tasks.workunit.client.0.trial116.stdout: "objects_scrubbed": 0, 2026-02-12T18:18:04.971 INFO:tasks.workunit.client.0.trial116.stdout: "log_size": 101, 2026-02-12T18:18:04.971 INFO:tasks.workunit.client.0.trial116.stdout: "log_dups_size": 200, 2026-02-12T18:18:04.971 INFO:tasks.workunit.client.0.trial116.stdout: "ondisk_log_size": 101, 2026-02-12T18:18:04.971 INFO:tasks.workunit.client.0.trial116.stdout: "stats_invalid": false, 2026-02-12T18:18:04.971 INFO:tasks.workunit.client.0.trial116.stdout: "dirty_stats_invalid": false, 2026-02-12T18:18:04.971 INFO:tasks.workunit.client.0.trial116.stdout: "omap_stats_invalid": false, 2026-02-12T18:18:04.971 INFO:tasks.workunit.client.0.trial116.stdout: "hitset_stats_invalid": false, 2026-02-12T18:18:04.971 INFO:tasks.workunit.client.0.trial116.stdout: "hitset_bytes_stats_invalid": false, 2026-02-12T18:18:04.972 INFO:tasks.workunit.client.0.trial116.stdout: "pin_stats_invalid": false, 2026-02-12T18:18:04.972 INFO:tasks.workunit.client.0.trial116.stdout: "manifest_stats_invalid": false, 2026-02-12T18:18:04.972 INFO:tasks.workunit.client.0.trial116.stdout: "snaptrimq_len": 0, 2026-02-12T18:18:04.972 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub_duration": 0, 2026-02-12T18:18:04.972 INFO:tasks.workunit.client.0.trial116.stdout: "scrub_schedule": "no scrub is scheduled", 2026-02-12T18:18:04.972 INFO:tasks.workunit.client.0.trial116.stdout: "scrub_duration": 0, 2026-02-12T18:18:04.972 INFO:tasks.workunit.client.0.trial116.stdout: "objects_trimmed": 0, 2026-02-12T18:18:04.972 INFO:tasks.workunit.client.0.trial116.stdout: "snaptrim_duration": 0, 2026-02-12T18:18:04.972 INFO:tasks.workunit.client.0.trial116.stdout: "stat_sum": { 2026-02-12T18:18:04.972 INFO:tasks.workunit.client.0.trial116.stdout: "num_bytes": 1232896, 2026-02-12T18:18:04.972 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects": 301, 2026-02-12T18:18:04.972 INFO:tasks.workunit.client.0.trial116.stdout: "num_object_clones": 0, 2026-02-12T18:18:04.973 INFO:tasks.workunit.client.0.trial116.stdout: "num_object_copies": 1505, 2026-02-12T18:18:04.973 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_missing_on_primary": 1, 2026-02-12T18:18:04.973 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_missing": 1, 2026-02-12T18:18:04.973 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_degraded": 303, 2026-02-12T18:18:04.973 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_misplaced": 0, 2026-02-12T18:18:04.973 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_unfound": 1, 2026-02-12T18:18:04.973 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_dirty": 301, 2026-02-12T18:18:04.973 INFO:tasks.workunit.client.0.trial116.stdout: "num_whiteouts": 0, 2026-02-12T18:18:04.973 INFO:tasks.workunit.client.0.trial116.stdout: "num_read": 0, 2026-02-12T18:18:04.973 INFO:tasks.workunit.client.0.trial116.stdout: "num_read_kb": 0, 2026-02-12T18:18:04.973 INFO:tasks.workunit.client.0.trial116.stdout: "num_write": 301, 2026-02-12T18:18:04.973 INFO:tasks.workunit.client.0.trial116.stdout: "num_write_kb": 1204, 2026-02-12T18:18:04.973 INFO:tasks.workunit.client.0.trial116.stdout: "num_scrub_errors": 0, 2026-02-12T18:18:04.974 INFO:tasks.workunit.client.0.trial116.stdout: "num_shallow_scrub_errors": 0, 2026-02-12T18:18:04.974 INFO:tasks.workunit.client.0.trial116.stdout: "num_deep_scrub_errors": 0, 2026-02-12T18:18:04.974 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_recovered": 0, 2026-02-12T18:18:04.974 INFO:tasks.workunit.client.0.trial116.stdout: "num_bytes_recovered": 0, 2026-02-12T18:18:04.974 INFO:tasks.workunit.client.0.trial116.stdout: "num_keys_recovered": 0, 2026-02-12T18:18:04.974 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_omap": 0, 2026-02-12T18:18:04.974 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_hit_set_archive": 0, 2026-02-12T18:18:04.974 INFO:tasks.workunit.client.0.trial116.stdout: "num_bytes_hit_set_archive": 0, 2026-02-12T18:18:04.974 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush": 0, 2026-02-12T18:18:04.974 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush_kb": 0, 2026-02-12T18:18:04.974 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict": 0, 2026-02-12T18:18:04.974 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict_kb": 0, 2026-02-12T18:18:04.974 INFO:tasks.workunit.client.0.trial116.stdout: "num_promote": 0, 2026-02-12T18:18:04.975 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush_mode_high": 0, 2026-02-12T18:18:04.975 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush_mode_low": 0, 2026-02-12T18:18:04.975 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict_mode_some": 0, 2026-02-12T18:18:04.975 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict_mode_full": 0, 2026-02-12T18:18:04.975 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_pinned": 0, 2026-02-12T18:18:04.975 INFO:tasks.workunit.client.0.trial116.stdout: "num_legacy_snapsets": 0, 2026-02-12T18:18:04.975 INFO:tasks.workunit.client.0.trial116.stdout: "num_large_omap_objects": 0, 2026-02-12T18:18:04.975 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_manifest": 0, 2026-02-12T18:18:04.975 INFO:tasks.workunit.client.0.trial116.stdout: "num_omap_bytes": 0, 2026-02-12T18:18:04.975 INFO:tasks.workunit.client.0.trial116.stdout: "num_omap_keys": 0, 2026-02-12T18:18:04.975 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_repaired": 0 2026-02-12T18:18:04.975 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:18:04.975 INFO:tasks.workunit.client.0.trial116.stdout: "up": [ 2026-02-12T18:18:04.975 INFO:tasks.workunit.client.0.trial116.stdout: 3, 2026-02-12T18:18:04.976 INFO:tasks.workunit.client.0.trial116.stdout: 1, 2026-02-12T18:18:04.976 INFO:tasks.workunit.client.0.trial116.stdout: 4, 2026-02-12T18:18:04.976 INFO:tasks.workunit.client.0.trial116.stdout: 0, 2026-02-12T18:18:04.976 INFO:tasks.workunit.client.0.trial116.stdout: 2147483647 2026-02-12T18:18:04.976 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:18:04.976 INFO:tasks.workunit.client.0.trial116.stdout: "acting": [ 2026-02-12T18:18:04.976 INFO:tasks.workunit.client.0.trial116.stdout: 3, 2026-02-12T18:18:04.976 INFO:tasks.workunit.client.0.trial116.stdout: 1, 2026-02-12T18:18:04.976 INFO:tasks.workunit.client.0.trial116.stdout: 4, 2026-02-12T18:18:04.976 INFO:tasks.workunit.client.0.trial116.stdout: 0, 2026-02-12T18:18:04.976 INFO:tasks.workunit.client.0.trial116.stdout: 2147483647 2026-02-12T18:18:04.976 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:18:04.977 INFO:tasks.workunit.client.0.trial116.stdout: "avail_no_missing": [ 2026-02-12T18:18:04.977 INFO:tasks.workunit.client.0.trial116.stdout: "0(3)", 2026-02-12T18:18:04.977 INFO:tasks.workunit.client.0.trial116.stdout: "4(2)" 2026-02-12T18:18:04.977 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:18:04.977 INFO:tasks.workunit.client.0.trial116.stdout: "object_location_counts": [ 2026-02-12T18:18:04.977 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:18:04.977 INFO:tasks.workunit.client.0.trial116.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-12T18:18:04.977 INFO:tasks.workunit.client.0.trial116.stdout: "objects": 300 2026-02-12T18:18:04.977 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:18:04.977 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:18:04.977 INFO:tasks.workunit.client.0.trial116.stdout: "shards": "0(3),4(2)", 2026-02-12T18:18:04.977 INFO:tasks.workunit.client.0.trial116.stdout: "objects": 1 2026-02-12T18:18:04.977 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:18:04.978 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:18:04.978 INFO:tasks.workunit.client.0.trial116.stdout: "blocked_by": [], 2026-02-12T18:18:04.978 INFO:tasks.workunit.client.0.trial116.stdout: "up_primary": 3, 2026-02-12T18:18:04.978 INFO:tasks.workunit.client.0.trial116.stdout: "acting_primary": 3, 2026-02-12T18:18:04.978 INFO:tasks.workunit.client.0.trial116.stdout: "purged_snaps": [] 2026-02-12T18:18:04.978 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:18:04.978 INFO:tasks.workunit.client.0.trial116.stdout: "empty": 0, 2026-02-12T18:18:04.978 INFO:tasks.workunit.client.0.trial116.stdout: "dne": 0, 2026-02-12T18:18:04.978 INFO:tasks.workunit.client.0.trial116.stdout: "incomplete": 0, 2026-02-12T18:18:04.978 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_started": 57, 2026-02-12T18:18:04.978 INFO:tasks.workunit.client.0.trial116.stdout: "hit_set_history": { 2026-02-12T18:18:04.978 INFO:tasks.workunit.client.0.trial116.stdout: "current_last_update": "0'0", 2026-02-12T18:18:04.978 INFO:tasks.workunit.client.0.trial116.stdout: "history": [] 2026-02-12T18:18:04.979 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:18:04.979 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:18:04.979 INFO:tasks.workunit.client.0.trial116.stdout: "peer_info": [ 2026-02-12T18:18:04.979 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:18:04.979 INFO:tasks.workunit.client.0.trial116.stdout: "peer": "0(3)", 2026-02-12T18:18:04.979 INFO:tasks.workunit.client.0.trial116.stdout: "pgid": "2.0s3", 2026-02-12T18:18:04.979 INFO:tasks.workunit.client.0.trial116.stdout: "shard": "3", 2026-02-12T18:18:04.979 INFO:tasks.workunit.client.0.trial116.stdout: "last_update": "47'301", 2026-02-12T18:18:04.979 INFO:tasks.workunit.client.0.trial116.stdout: "last_complete": "47'301", 2026-02-12T18:18:04.979 INFO:tasks.workunit.client.0.trial116.stdout: "log_tail": "47'200", 2026-02-12T18:18:04.979 INFO:tasks.workunit.client.0.trial116.stdout: "last_user_version": 301, 2026-02-12T18:18:04.979 INFO:tasks.workunit.client.0.trial116.stdout: "last_backfill": "MAX", 2026-02-12T18:18:04.979 INFO:tasks.workunit.client.0.trial116.stdout: "partial_writes_last_complete": [], 2026-02-12T18:18:04.979 INFO:tasks.workunit.client.0.trial116.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-12T18:18:04.980 INFO:tasks.workunit.client.0.trial116.stdout: "purged_snaps": [], 2026-02-12T18:18:04.980 INFO:tasks.workunit.client.0.trial116.stdout: "history": { 2026-02-12T18:18:04.980 INFO:tasks.workunit.client.0.trial116.stdout: "epoch_created": 41, 2026-02-12T18:18:04.980 INFO:tasks.workunit.client.0.trial116.stdout: "epoch_pool_created": 41, 2026-02-12T18:18:04.980 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_started": 57, 2026-02-12T18:18:04.980 INFO:tasks.workunit.client.0.trial116.stdout: "last_interval_started": 56, 2026-02-12T18:18:04.980 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_clean": 42, 2026-02-12T18:18:04.980 INFO:tasks.workunit.client.0.trial116.stdout: "last_interval_clean": 41, 2026-02-12T18:18:04.980 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_split": 0, 2026-02-12T18:18:04.980 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_marked_full": 0, 2026-02-12T18:18:04.980 INFO:tasks.workunit.client.0.trial116.stdout: "same_up_since": 56, 2026-02-12T18:18:04.980 INFO:tasks.workunit.client.0.trial116.stdout: "same_interval_since": 56, 2026-02-12T18:18:04.980 INFO:tasks.workunit.client.0.trial116.stdout: "same_primary_since": 41, 2026-02-12T18:18:04.981 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub": "0'0", 2026-02-12T18:18:04.981 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub_stamp": "2026-02-12T18:17:29.908380+0000", 2026-02-12T18:18:04.981 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub": "0'0", 2026-02-12T18:18:04.981 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub_stamp": "2026-02-12T18:17:29.908380+0000", 2026-02-12T18:18:04.981 INFO:tasks.workunit.client.0.trial116.stdout: "last_clean_scrub_stamp": "2026-02-12T18:17:29.908380+0000", 2026-02-12T18:18:04.981 INFO:tasks.workunit.client.0.trial116.stdout: "prior_readable_until_ub": 0 2026-02-12T18:18:04.981 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:18:04.981 INFO:tasks.workunit.client.0.trial116.stdout: "stats": { 2026-02-12T18:18:04.981 INFO:tasks.workunit.client.0.trial116.stdout: "version": "47'301", 2026-02-12T18:18:04.981 INFO:tasks.workunit.client.0.trial116.stdout: "reported_seq": 910, 2026-02-12T18:18:04.981 INFO:tasks.workunit.client.0.trial116.stdout: "reported_epoch": 56, 2026-02-12T18:18:04.981 INFO:tasks.workunit.client.0.trial116.stdout: "state": "active+remapped", 2026-02-12T18:18:04.981 INFO:tasks.workunit.client.0.trial116.stdout: "last_fresh": "2026-02-12T18:17:41.445658+0000", 2026-02-12T18:18:04.982 INFO:tasks.workunit.client.0.trial116.stdout: "last_change": "2026-02-12T18:17:35.718463+0000", 2026-02-12T18:18:04.982 INFO:tasks.workunit.client.0.trial116.stdout: "last_active": "2026-02-12T18:17:41.445658+0000", 2026-02-12T18:18:04.982 INFO:tasks.workunit.client.0.trial116.stdout: "last_peered": "2026-02-12T18:17:41.445658+0000", 2026-02-12T18:18:04.982 INFO:tasks.workunit.client.0.trial116.stdout: "last_clean": "2026-02-12T18:17:35.094275+0000", 2026-02-12T18:18:04.982 INFO:tasks.workunit.client.0.trial116.stdout: "last_became_active": "2026-02-12T18:17:35.718463+0000", 2026-02-12T18:18:04.982 INFO:tasks.workunit.client.0.trial116.stdout: "last_became_peered": "2026-02-12T18:17:35.718463+0000", 2026-02-12T18:18:04.982 INFO:tasks.workunit.client.0.trial116.stdout: "last_unstale": "2026-02-12T18:17:41.445658+0000", 2026-02-12T18:18:04.982 INFO:tasks.workunit.client.0.trial116.stdout: "last_undegraded": "2026-02-12T18:17:35.717261+0000", 2026-02-12T18:18:04.982 INFO:tasks.workunit.client.0.trial116.stdout: "last_fullsized": "2026-02-12T18:17:35.717178+0000", 2026-02-12T18:18:04.982 INFO:tasks.workunit.client.0.trial116.stdout: "mapping_epoch": 56, 2026-02-12T18:18:04.982 INFO:tasks.workunit.client.0.trial116.stdout: "log_start": "47'200", 2026-02-12T18:18:04.982 INFO:tasks.workunit.client.0.trial116.stdout: "ondisk_log_start": "47'200", 2026-02-12T18:18:04.983 INFO:tasks.workunit.client.0.trial116.stdout: "created": 41, 2026-02-12T18:18:04.983 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_clean": 42, 2026-02-12T18:18:04.983 INFO:tasks.workunit.client.0.trial116.stdout: "parent": "0.0", 2026-02-12T18:18:04.983 INFO:tasks.workunit.client.0.trial116.stdout: "parent_split_bits": 0, 2026-02-12T18:18:04.983 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub": "0'0", 2026-02-12T18:18:04.983 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub_stamp": "2026-02-12T18:17:29.908380+0000", 2026-02-12T18:18:04.983 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub": "0'0", 2026-02-12T18:18:04.983 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub_stamp": "2026-02-12T18:17:29.908380+0000", 2026-02-12T18:18:04.983 INFO:tasks.workunit.client.0.trial116.stdout: "last_clean_scrub_stamp": "2026-02-12T18:17:29.908380+0000", 2026-02-12T18:18:04.983 INFO:tasks.workunit.client.0.trial116.stdout: "objects_scrubbed": 0, 2026-02-12T18:18:04.983 INFO:tasks.workunit.client.0.trial116.stdout: "log_size": 101, 2026-02-12T18:18:04.983 INFO:tasks.workunit.client.0.trial116.stdout: "log_dups_size": 200, 2026-02-12T18:18:04.983 INFO:tasks.workunit.client.0.trial116.stdout: "ondisk_log_size": 101, 2026-02-12T18:18:04.984 INFO:tasks.workunit.client.0.trial116.stdout: "stats_invalid": false, 2026-02-12T18:18:04.984 INFO:tasks.workunit.client.0.trial116.stdout: "dirty_stats_invalid": false, 2026-02-12T18:18:04.984 INFO:tasks.workunit.client.0.trial116.stdout: "omap_stats_invalid": false, 2026-02-12T18:18:04.984 INFO:tasks.workunit.client.0.trial116.stdout: "hitset_stats_invalid": false, 2026-02-12T18:18:04.984 INFO:tasks.workunit.client.0.trial116.stdout: "hitset_bytes_stats_invalid": false, 2026-02-12T18:18:04.984 INFO:tasks.workunit.client.0.trial116.stdout: "pin_stats_invalid": false, 2026-02-12T18:18:04.984 INFO:tasks.workunit.client.0.trial116.stdout: "manifest_stats_invalid": false, 2026-02-12T18:18:04.984 INFO:tasks.workunit.client.0.trial116.stdout: "snaptrimq_len": 0, 2026-02-12T18:18:04.984 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub_duration": 0, 2026-02-12T18:18:04.984 INFO:tasks.workunit.client.0.trial116.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-13T21:41:30.320669+0000", 2026-02-12T18:18:04.984 INFO:tasks.workunit.client.0.trial116.stdout: "scrub_duration": 0, 2026-02-12T18:18:04.984 INFO:tasks.workunit.client.0.trial116.stdout: "objects_trimmed": 0, 2026-02-12T18:18:04.984 INFO:tasks.workunit.client.0.trial116.stdout: "snaptrim_duration": 0, 2026-02-12T18:18:04.985 INFO:tasks.workunit.client.0.trial116.stdout: "stat_sum": { 2026-02-12T18:18:04.985 INFO:tasks.workunit.client.0.trial116.stdout: "num_bytes": 1232896, 2026-02-12T18:18:04.985 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects": 301, 2026-02-12T18:18:04.985 INFO:tasks.workunit.client.0.trial116.stdout: "num_object_clones": 0, 2026-02-12T18:18:04.985 INFO:tasks.workunit.client.0.trial116.stdout: "num_object_copies": 1505, 2026-02-12T18:18:04.985 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_missing_on_primary": 0, 2026-02-12T18:18:04.985 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_missing": 0, 2026-02-12T18:18:04.985 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_degraded": 301, 2026-02-12T18:18:04.985 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_misplaced": 0, 2026-02-12T18:18:04.985 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_unfound": 0, 2026-02-12T18:18:04.985 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_dirty": 301, 2026-02-12T18:18:04.985 INFO:tasks.workunit.client.0.trial116.stdout: "num_whiteouts": 0, 2026-02-12T18:18:04.985 INFO:tasks.workunit.client.0.trial116.stdout: "num_read": 0, 2026-02-12T18:18:04.986 INFO:tasks.workunit.client.0.trial116.stdout: "num_read_kb": 0, 2026-02-12T18:18:04.986 INFO:tasks.workunit.client.0.trial116.stdout: "num_write": 301, 2026-02-12T18:18:04.986 INFO:tasks.workunit.client.0.trial116.stdout: "num_write_kb": 1204, 2026-02-12T18:18:04.986 INFO:tasks.workunit.client.0.trial116.stdout: "num_scrub_errors": 0, 2026-02-12T18:18:04.986 INFO:tasks.workunit.client.0.trial116.stdout: "num_shallow_scrub_errors": 0, 2026-02-12T18:18:04.986 INFO:tasks.workunit.client.0.trial116.stdout: "num_deep_scrub_errors": 0, 2026-02-12T18:18:04.986 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_recovered": 0, 2026-02-12T18:18:04.986 INFO:tasks.workunit.client.0.trial116.stdout: "num_bytes_recovered": 0, 2026-02-12T18:18:04.986 INFO:tasks.workunit.client.0.trial116.stdout: "num_keys_recovered": 0, 2026-02-12T18:18:04.986 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_omap": 0, 2026-02-12T18:18:04.986 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_hit_set_archive": 0, 2026-02-12T18:18:04.986 INFO:tasks.workunit.client.0.trial116.stdout: "num_bytes_hit_set_archive": 0, 2026-02-12T18:18:04.986 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush": 0, 2026-02-12T18:18:04.986 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush_kb": 0, 2026-02-12T18:18:04.987 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict": 0, 2026-02-12T18:18:04.987 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict_kb": 0, 2026-02-12T18:18:04.987 INFO:tasks.workunit.client.0.trial116.stdout: "num_promote": 0, 2026-02-12T18:18:04.987 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush_mode_high": 0, 2026-02-12T18:18:04.987 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush_mode_low": 0, 2026-02-12T18:18:04.987 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict_mode_some": 0, 2026-02-12T18:18:04.987 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict_mode_full": 0, 2026-02-12T18:18:04.987 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_pinned": 0, 2026-02-12T18:18:04.987 INFO:tasks.workunit.client.0.trial116.stdout: "num_legacy_snapsets": 0, 2026-02-12T18:18:04.987 INFO:tasks.workunit.client.0.trial116.stdout: "num_large_omap_objects": 0, 2026-02-12T18:18:04.987 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_manifest": 0, 2026-02-12T18:18:04.987 INFO:tasks.workunit.client.0.trial116.stdout: "num_omap_bytes": 0, 2026-02-12T18:18:04.987 INFO:tasks.workunit.client.0.trial116.stdout: "num_omap_keys": 0, 2026-02-12T18:18:04.988 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_repaired": 0 2026-02-12T18:18:04.988 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:18:04.988 INFO:tasks.workunit.client.0.trial116.stdout: "up": [ 2026-02-12T18:18:04.988 INFO:tasks.workunit.client.0.trial116.stdout: 3, 2026-02-12T18:18:04.988 INFO:tasks.workunit.client.0.trial116.stdout: 1, 2026-02-12T18:18:04.988 INFO:tasks.workunit.client.0.trial116.stdout: 4, 2026-02-12T18:18:04.988 INFO:tasks.workunit.client.0.trial116.stdout: 0, 2026-02-12T18:18:04.988 INFO:tasks.workunit.client.0.trial116.stdout: 2147483647 2026-02-12T18:18:04.988 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:18:04.988 INFO:tasks.workunit.client.0.trial116.stdout: "acting": [ 2026-02-12T18:18:04.988 INFO:tasks.workunit.client.0.trial116.stdout: 3, 2026-02-12T18:18:04.988 INFO:tasks.workunit.client.0.trial116.stdout: 1, 2026-02-12T18:18:04.988 INFO:tasks.workunit.client.0.trial116.stdout: 4, 2026-02-12T18:18:04.989 INFO:tasks.workunit.client.0.trial116.stdout: 0, 2026-02-12T18:18:04.989 INFO:tasks.workunit.client.0.trial116.stdout: 2147483647 2026-02-12T18:18:04.989 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:18:04.989 INFO:tasks.workunit.client.0.trial116.stdout: "avail_no_missing": [ 2026-02-12T18:18:04.989 INFO:tasks.workunit.client.0.trial116.stdout: "3(0)", 2026-02-12T18:18:04.989 INFO:tasks.workunit.client.0.trial116.stdout: "0(3)", 2026-02-12T18:18:04.989 INFO:tasks.workunit.client.0.trial116.stdout: "1(1)", 2026-02-12T18:18:04.989 INFO:tasks.workunit.client.0.trial116.stdout: "4(2)" 2026-02-12T18:18:04.989 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:18:04.989 INFO:tasks.workunit.client.0.trial116.stdout: "object_location_counts": [ 2026-02-12T18:18:04.989 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:18:04.989 INFO:tasks.workunit.client.0.trial116.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-12T18:18:04.989 INFO:tasks.workunit.client.0.trial116.stdout: "objects": 301 2026-02-12T18:18:04.990 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:18:04.990 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:18:04.990 INFO:tasks.workunit.client.0.trial116.stdout: "blocked_by": [], 2026-02-12T18:18:04.990 INFO:tasks.workunit.client.0.trial116.stdout: "up_primary": 3, 2026-02-12T18:18:04.990 INFO:tasks.workunit.client.0.trial116.stdout: "acting_primary": 3, 2026-02-12T18:18:04.990 INFO:tasks.workunit.client.0.trial116.stdout: "purged_snaps": [] 2026-02-12T18:18:04.990 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:18:04.990 INFO:tasks.workunit.client.0.trial116.stdout: "empty": 0, 2026-02-12T18:18:04.990 INFO:tasks.workunit.client.0.trial116.stdout: "dne": 0, 2026-02-12T18:18:04.990 INFO:tasks.workunit.client.0.trial116.stdout: "incomplete": 0, 2026-02-12T18:18:04.990 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_started": 57, 2026-02-12T18:18:04.990 INFO:tasks.workunit.client.0.trial116.stdout: "hit_set_history": { 2026-02-12T18:18:04.990 INFO:tasks.workunit.client.0.trial116.stdout: "current_last_update": "0'0", 2026-02-12T18:18:04.991 INFO:tasks.workunit.client.0.trial116.stdout: "history": [] 2026-02-12T18:18:04.991 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:18:04.991 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:18:04.991 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:18:04.991 INFO:tasks.workunit.client.0.trial116.stdout: "peer": "1(1)", 2026-02-12T18:18:04.991 INFO:tasks.workunit.client.0.trial116.stdout: "pgid": "2.0s1", 2026-02-12T18:18:04.991 INFO:tasks.workunit.client.0.trial116.stdout: "shard": "1", 2026-02-12T18:18:04.991 INFO:tasks.workunit.client.0.trial116.stdout: "last_update": "47'301", 2026-02-12T18:18:04.991 INFO:tasks.workunit.client.0.trial116.stdout: "last_complete": "47'301", 2026-02-12T18:18:04.991 INFO:tasks.workunit.client.0.trial116.stdout: "log_tail": "47'200", 2026-02-12T18:18:04.991 INFO:tasks.workunit.client.0.trial116.stdout: "last_user_version": 301, 2026-02-12T18:18:04.991 INFO:tasks.workunit.client.0.trial116.stdout: "last_backfill": "MAX", 2026-02-12T18:18:04.991 INFO:tasks.workunit.client.0.trial116.stdout: "partial_writes_last_complete": [], 2026-02-12T18:18:04.992 INFO:tasks.workunit.client.0.trial116.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-12T18:18:04.992 INFO:tasks.workunit.client.0.trial116.stdout: "purged_snaps": [], 2026-02-12T18:18:04.992 INFO:tasks.workunit.client.0.trial116.stdout: "history": { 2026-02-12T18:18:04.992 INFO:tasks.workunit.client.0.trial116.stdout: "epoch_created": 41, 2026-02-12T18:18:04.992 INFO:tasks.workunit.client.0.trial116.stdout: "epoch_pool_created": 41, 2026-02-12T18:18:04.992 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_started": 57, 2026-02-12T18:18:04.992 INFO:tasks.workunit.client.0.trial116.stdout: "last_interval_started": 56, 2026-02-12T18:18:04.992 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_clean": 42, 2026-02-12T18:18:04.992 INFO:tasks.workunit.client.0.trial116.stdout: "last_interval_clean": 41, 2026-02-12T18:18:04.992 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_split": 0, 2026-02-12T18:18:04.992 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_marked_full": 0, 2026-02-12T18:18:04.992 INFO:tasks.workunit.client.0.trial116.stdout: "same_up_since": 56, 2026-02-12T18:18:04.992 INFO:tasks.workunit.client.0.trial116.stdout: "same_interval_since": 56, 2026-02-12T18:18:04.992 INFO:tasks.workunit.client.0.trial116.stdout: "same_primary_since": 41, 2026-02-12T18:18:04.993 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub": "0'0", 2026-02-12T18:18:04.993 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub_stamp": "2026-02-12T18:17:29.908380+0000", 2026-02-12T18:18:04.993 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub": "0'0", 2026-02-12T18:18:04.993 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub_stamp": "2026-02-12T18:17:29.908380+0000", 2026-02-12T18:18:04.993 INFO:tasks.workunit.client.0.trial116.stdout: "last_clean_scrub_stamp": "2026-02-12T18:17:29.908380+0000", 2026-02-12T18:18:04.993 INFO:tasks.workunit.client.0.trial116.stdout: "prior_readable_until_ub": 0 2026-02-12T18:18:04.993 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:18:04.993 INFO:tasks.workunit.client.0.trial116.stdout: "stats": { 2026-02-12T18:18:04.993 INFO:tasks.workunit.client.0.trial116.stdout: "version": "47'301", 2026-02-12T18:18:04.993 INFO:tasks.workunit.client.0.trial116.stdout: "reported_seq": 331, 2026-02-12T18:18:04.993 INFO:tasks.workunit.client.0.trial116.stdout: "reported_epoch": 47, 2026-02-12T18:18:04.993 INFO:tasks.workunit.client.0.trial116.stdout: "state": "active+remapped", 2026-02-12T18:18:04.993 INFO:tasks.workunit.client.0.trial116.stdout: "last_fresh": "2026-02-12T18:17:41.445658+0000", 2026-02-12T18:18:04.994 INFO:tasks.workunit.client.0.trial116.stdout: "last_change": "2026-02-12T18:17:35.718463+0000", 2026-02-12T18:18:04.994 INFO:tasks.workunit.client.0.trial116.stdout: "last_active": "2026-02-12T18:17:41.445658+0000", 2026-02-12T18:18:04.994 INFO:tasks.workunit.client.0.trial116.stdout: "last_peered": "2026-02-12T18:17:41.445658+0000", 2026-02-12T18:18:04.994 INFO:tasks.workunit.client.0.trial116.stdout: "last_clean": "2026-02-12T18:17:35.094275+0000", 2026-02-12T18:18:04.994 INFO:tasks.workunit.client.0.trial116.stdout: "last_became_active": "2026-02-12T18:17:35.718463+0000", 2026-02-12T18:18:04.994 INFO:tasks.workunit.client.0.trial116.stdout: "last_became_peered": "2026-02-12T18:17:35.718463+0000", 2026-02-12T18:18:04.994 INFO:tasks.workunit.client.0.trial116.stdout: "last_unstale": "2026-02-12T18:17:41.445658+0000", 2026-02-12T18:18:04.994 INFO:tasks.workunit.client.0.trial116.stdout: "last_undegraded": "2026-02-12T18:17:35.717261+0000", 2026-02-12T18:18:04.994 INFO:tasks.workunit.client.0.trial116.stdout: "last_fullsized": "2026-02-12T18:17:35.717178+0000", 2026-02-12T18:18:04.994 INFO:tasks.workunit.client.0.trial116.stdout: "mapping_epoch": 56, 2026-02-12T18:18:04.994 INFO:tasks.workunit.client.0.trial116.stdout: "log_start": "47'200", 2026-02-12T18:18:04.994 INFO:tasks.workunit.client.0.trial116.stdout: "ondisk_log_start": "47'200", 2026-02-12T18:18:04.994 INFO:tasks.workunit.client.0.trial116.stdout: "created": 41, 2026-02-12T18:18:04.994 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_clean": 42, 2026-02-12T18:18:04.995 INFO:tasks.workunit.client.0.trial116.stdout: "parent": "0.0", 2026-02-12T18:18:04.995 INFO:tasks.workunit.client.0.trial116.stdout: "parent_split_bits": 0, 2026-02-12T18:18:04.995 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub": "0'0", 2026-02-12T18:18:04.995 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub_stamp": "2026-02-12T18:17:29.908380+0000", 2026-02-12T18:18:04.995 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub": "0'0", 2026-02-12T18:18:04.995 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub_stamp": "2026-02-12T18:17:29.908380+0000", 2026-02-12T18:18:04.995 INFO:tasks.workunit.client.0.trial116.stdout: "last_clean_scrub_stamp": "2026-02-12T18:17:29.908380+0000", 2026-02-12T18:18:04.995 INFO:tasks.workunit.client.0.trial116.stdout: "objects_scrubbed": 0, 2026-02-12T18:18:04.995 INFO:tasks.workunit.client.0.trial116.stdout: "log_size": 101, 2026-02-12T18:18:04.995 INFO:tasks.workunit.client.0.trial116.stdout: "log_dups_size": 200, 2026-02-12T18:18:04.995 INFO:tasks.workunit.client.0.trial116.stdout: "ondisk_log_size": 101, 2026-02-12T18:18:04.995 INFO:tasks.workunit.client.0.trial116.stdout: "stats_invalid": false, 2026-02-12T18:18:04.995 INFO:tasks.workunit.client.0.trial116.stdout: "dirty_stats_invalid": false, 2026-02-12T18:18:04.995 INFO:tasks.workunit.client.0.trial116.stdout: "omap_stats_invalid": false, 2026-02-12T18:18:04.996 INFO:tasks.workunit.client.0.trial116.stdout: "hitset_stats_invalid": false, 2026-02-12T18:18:04.996 INFO:tasks.workunit.client.0.trial116.stdout: "hitset_bytes_stats_invalid": false, 2026-02-12T18:18:04.996 INFO:tasks.workunit.client.0.trial116.stdout: "pin_stats_invalid": false, 2026-02-12T18:18:04.996 INFO:tasks.workunit.client.0.trial116.stdout: "manifest_stats_invalid": false, 2026-02-12T18:18:04.996 INFO:tasks.workunit.client.0.trial116.stdout: "snaptrimq_len": 0, 2026-02-12T18:18:04.996 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub_duration": 0, 2026-02-12T18:18:04.996 INFO:tasks.workunit.client.0.trial116.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-13T21:41:30.320669+0000", 2026-02-12T18:18:04.996 INFO:tasks.workunit.client.0.trial116.stdout: "scrub_duration": 0, 2026-02-12T18:18:04.996 INFO:tasks.workunit.client.0.trial116.stdout: "objects_trimmed": 0, 2026-02-12T18:18:04.996 INFO:tasks.workunit.client.0.trial116.stdout: "snaptrim_duration": 0, 2026-02-12T18:18:04.996 INFO:tasks.workunit.client.0.trial116.stdout: "stat_sum": { 2026-02-12T18:18:04.996 INFO:tasks.workunit.client.0.trial116.stdout: "num_bytes": 1232896, 2026-02-12T18:18:04.996 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects": 301, 2026-02-12T18:18:04.997 INFO:tasks.workunit.client.0.trial116.stdout: "num_object_clones": 0, 2026-02-12T18:18:04.997 INFO:tasks.workunit.client.0.trial116.stdout: "num_object_copies": 1505, 2026-02-12T18:18:04.997 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_missing_on_primary": 0, 2026-02-12T18:18:04.997 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_missing": 1, 2026-02-12T18:18:04.997 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_degraded": 301, 2026-02-12T18:18:04.997 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_misplaced": 0, 2026-02-12T18:18:04.997 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_unfound": 0, 2026-02-12T18:18:04.997 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_dirty": 301, 2026-02-12T18:18:04.997 INFO:tasks.workunit.client.0.trial116.stdout: "num_whiteouts": 0, 2026-02-12T18:18:04.997 INFO:tasks.workunit.client.0.trial116.stdout: "num_read": 0, 2026-02-12T18:18:04.997 INFO:tasks.workunit.client.0.trial116.stdout: "num_read_kb": 0, 2026-02-12T18:18:04.997 INFO:tasks.workunit.client.0.trial116.stdout: "num_write": 301, 2026-02-12T18:18:04.997 INFO:tasks.workunit.client.0.trial116.stdout: "num_write_kb": 1204, 2026-02-12T18:18:04.997 INFO:tasks.workunit.client.0.trial116.stdout: "num_scrub_errors": 0, 2026-02-12T18:18:04.998 INFO:tasks.workunit.client.0.trial116.stdout: "num_shallow_scrub_errors": 0, 2026-02-12T18:18:04.998 INFO:tasks.workunit.client.0.trial116.stdout: "num_deep_scrub_errors": 0, 2026-02-12T18:18:04.998 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_recovered": 0, 2026-02-12T18:18:04.998 INFO:tasks.workunit.client.0.trial116.stdout: "num_bytes_recovered": 0, 2026-02-12T18:18:04.998 INFO:tasks.workunit.client.0.trial116.stdout: "num_keys_recovered": 0, 2026-02-12T18:18:04.998 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_omap": 0, 2026-02-12T18:18:04.998 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_hit_set_archive": 0, 2026-02-12T18:18:04.998 INFO:tasks.workunit.client.0.trial116.stdout: "num_bytes_hit_set_archive": 0, 2026-02-12T18:18:04.998 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush": 0, 2026-02-12T18:18:04.998 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush_kb": 0, 2026-02-12T18:18:04.998 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict": 0, 2026-02-12T18:18:04.998 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict_kb": 0, 2026-02-12T18:18:04.998 INFO:tasks.workunit.client.0.trial116.stdout: "num_promote": 0, 2026-02-12T18:18:04.998 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush_mode_high": 0, 2026-02-12T18:18:04.999 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush_mode_low": 0, 2026-02-12T18:18:04.999 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict_mode_some": 0, 2026-02-12T18:18:04.999 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict_mode_full": 0, 2026-02-12T18:18:04.999 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_pinned": 0, 2026-02-12T18:18:04.999 INFO:tasks.workunit.client.0.trial116.stdout: "num_legacy_snapsets": 0, 2026-02-12T18:18:04.999 INFO:tasks.workunit.client.0.trial116.stdout: "num_large_omap_objects": 0, 2026-02-12T18:18:04.999 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_manifest": 0, 2026-02-12T18:18:04.999 INFO:tasks.workunit.client.0.trial116.stdout: "num_omap_bytes": 0, 2026-02-12T18:18:04.999 INFO:tasks.workunit.client.0.trial116.stdout: "num_omap_keys": 0, 2026-02-12T18:18:04.999 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_repaired": 0 2026-02-12T18:18:04.999 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:18:04.999 INFO:tasks.workunit.client.0.trial116.stdout: "up": [ 2026-02-12T18:18:04.999 INFO:tasks.workunit.client.0.trial116.stdout: 3, 2026-02-12T18:18:05.000 INFO:tasks.workunit.client.0.trial116.stdout: 1, 2026-02-12T18:18:05.000 INFO:tasks.workunit.client.0.trial116.stdout: 4, 2026-02-12T18:18:05.000 INFO:tasks.workunit.client.0.trial116.stdout: 0, 2026-02-12T18:18:05.000 INFO:tasks.workunit.client.0.trial116.stdout: 2147483647 2026-02-12T18:18:05.000 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:18:05.000 INFO:tasks.workunit.client.0.trial116.stdout: "acting": [ 2026-02-12T18:18:05.000 INFO:tasks.workunit.client.0.trial116.stdout: 3, 2026-02-12T18:18:05.000 INFO:tasks.workunit.client.0.trial116.stdout: 1, 2026-02-12T18:18:05.000 INFO:tasks.workunit.client.0.trial116.stdout: 4, 2026-02-12T18:18:05.000 INFO:tasks.workunit.client.0.trial116.stdout: 0, 2026-02-12T18:18:05.000 INFO:tasks.workunit.client.0.trial116.stdout: 2147483647 2026-02-12T18:18:05.000 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:18:05.000 INFO:tasks.workunit.client.0.trial116.stdout: "avail_no_missing": [ 2026-02-12T18:18:05.000 INFO:tasks.workunit.client.0.trial116.stdout: "3(0)", 2026-02-12T18:18:05.000 INFO:tasks.workunit.client.0.trial116.stdout: "0(3)", 2026-02-12T18:18:05.001 INFO:tasks.workunit.client.0.trial116.stdout: "1(1)", 2026-02-12T18:18:05.001 INFO:tasks.workunit.client.0.trial116.stdout: "4(2)" 2026-02-12T18:18:05.001 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:18:05.001 INFO:tasks.workunit.client.0.trial116.stdout: "object_location_counts": [ 2026-02-12T18:18:05.001 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:18:05.001 INFO:tasks.workunit.client.0.trial116.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-12T18:18:05.001 INFO:tasks.workunit.client.0.trial116.stdout: "objects": 301 2026-02-12T18:18:05.001 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:18:05.001 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:18:05.001 INFO:tasks.workunit.client.0.trial116.stdout: "blocked_by": [], 2026-02-12T18:18:05.001 INFO:tasks.workunit.client.0.trial116.stdout: "up_primary": 3, 2026-02-12T18:18:05.001 INFO:tasks.workunit.client.0.trial116.stdout: "acting_primary": 3, 2026-02-12T18:18:05.001 INFO:tasks.workunit.client.0.trial116.stdout: "purged_snaps": [] 2026-02-12T18:18:05.001 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:18:05.001 INFO:tasks.workunit.client.0.trial116.stdout: "empty": 0, 2026-02-12T18:18:05.002 INFO:tasks.workunit.client.0.trial116.stdout: "dne": 0, 2026-02-12T18:18:05.002 INFO:tasks.workunit.client.0.trial116.stdout: "incomplete": 0, 2026-02-12T18:18:05.002 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_started": 57, 2026-02-12T18:18:05.002 INFO:tasks.workunit.client.0.trial116.stdout: "hit_set_history": { 2026-02-12T18:18:05.002 INFO:tasks.workunit.client.0.trial116.stdout: "current_last_update": "0'0", 2026-02-12T18:18:05.002 INFO:tasks.workunit.client.0.trial116.stdout: "history": [] 2026-02-12T18:18:05.002 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:18:05.002 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:18:05.002 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:18:05.002 INFO:tasks.workunit.client.0.trial116.stdout: "peer": "4(2)", 2026-02-12T18:18:05.002 INFO:tasks.workunit.client.0.trial116.stdout: "pgid": "2.0s2", 2026-02-12T18:18:05.002 INFO:tasks.workunit.client.0.trial116.stdout: "shard": "2", 2026-02-12T18:18:05.002 INFO:tasks.workunit.client.0.trial116.stdout: "last_update": "47'301", 2026-02-12T18:18:05.002 INFO:tasks.workunit.client.0.trial116.stdout: "last_complete": "47'301", 2026-02-12T18:18:05.002 INFO:tasks.workunit.client.0.trial116.stdout: "log_tail": "47'200", 2026-02-12T18:18:05.002 INFO:tasks.workunit.client.0.trial116.stdout: "last_user_version": 301, 2026-02-12T18:18:05.003 INFO:tasks.workunit.client.0.trial116.stdout: "last_backfill": "MAX", 2026-02-12T18:18:05.003 INFO:tasks.workunit.client.0.trial116.stdout: "partial_writes_last_complete": [], 2026-02-12T18:18:05.003 INFO:tasks.workunit.client.0.trial116.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-12T18:18:05.003 INFO:tasks.workunit.client.0.trial116.stdout: "purged_snaps": [], 2026-02-12T18:18:05.003 INFO:tasks.workunit.client.0.trial116.stdout: "history": { 2026-02-12T18:18:05.003 INFO:tasks.workunit.client.0.trial116.stdout: "epoch_created": 41, 2026-02-12T18:18:05.003 INFO:tasks.workunit.client.0.trial116.stdout: "epoch_pool_created": 41, 2026-02-12T18:18:05.003 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_started": 57, 2026-02-12T18:18:05.003 INFO:tasks.workunit.client.0.trial116.stdout: "last_interval_started": 56, 2026-02-12T18:18:05.003 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_clean": 42, 2026-02-12T18:18:05.003 INFO:tasks.workunit.client.0.trial116.stdout: "last_interval_clean": 41, 2026-02-12T18:18:05.003 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_split": 0, 2026-02-12T18:18:05.003 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_marked_full": 0, 2026-02-12T18:18:05.003 INFO:tasks.workunit.client.0.trial116.stdout: "same_up_since": 56, 2026-02-12T18:18:05.003 INFO:tasks.workunit.client.0.trial116.stdout: "same_interval_since": 56, 2026-02-12T18:18:05.003 INFO:tasks.workunit.client.0.trial116.stdout: "same_primary_since": 41, 2026-02-12T18:18:05.004 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub": "0'0", 2026-02-12T18:18:05.004 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub_stamp": "2026-02-12T18:17:29.908380+0000", 2026-02-12T18:18:05.004 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub": "0'0", 2026-02-12T18:18:05.004 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub_stamp": "2026-02-12T18:17:29.908380+0000", 2026-02-12T18:18:05.004 INFO:tasks.workunit.client.0.trial116.stdout: "last_clean_scrub_stamp": "2026-02-12T18:17:29.908380+0000", 2026-02-12T18:18:05.004 INFO:tasks.workunit.client.0.trial116.stdout: "prior_readable_until_ub": 0 2026-02-12T18:18:05.004 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:18:05.004 INFO:tasks.workunit.client.0.trial116.stdout: "stats": { 2026-02-12T18:18:05.004 INFO:tasks.workunit.client.0.trial116.stdout: "version": "47'301", 2026-02-12T18:18:05.004 INFO:tasks.workunit.client.0.trial116.stdout: "reported_seq": 331, 2026-02-12T18:18:05.004 INFO:tasks.workunit.client.0.trial116.stdout: "reported_epoch": 47, 2026-02-12T18:18:05.004 INFO:tasks.workunit.client.0.trial116.stdout: "state": "active+remapped", 2026-02-12T18:18:05.004 INFO:tasks.workunit.client.0.trial116.stdout: "last_fresh": "2026-02-12T18:17:41.445658+0000", 2026-02-12T18:18:05.004 INFO:tasks.workunit.client.0.trial116.stdout: "last_change": "2026-02-12T18:17:35.718463+0000", 2026-02-12T18:18:05.004 INFO:tasks.workunit.client.0.trial116.stdout: "last_active": "2026-02-12T18:17:41.445658+0000", 2026-02-12T18:18:05.004 INFO:tasks.workunit.client.0.trial116.stdout: "last_peered": "2026-02-12T18:17:41.445658+0000", 2026-02-12T18:18:05.004 INFO:tasks.workunit.client.0.trial116.stdout: "last_clean": "2026-02-12T18:17:35.094275+0000", 2026-02-12T18:18:05.004 INFO:tasks.workunit.client.0.trial116.stdout: "last_became_active": "2026-02-12T18:17:35.718463+0000", 2026-02-12T18:18:05.005 INFO:tasks.workunit.client.0.trial116.stdout: "last_became_peered": "2026-02-12T18:17:35.718463+0000", 2026-02-12T18:18:05.005 INFO:tasks.workunit.client.0.trial116.stdout: "last_unstale": "2026-02-12T18:17:41.445658+0000", 2026-02-12T18:18:05.005 INFO:tasks.workunit.client.0.trial116.stdout: "last_undegraded": "2026-02-12T18:17:35.717261+0000", 2026-02-12T18:18:05.005 INFO:tasks.workunit.client.0.trial116.stdout: "last_fullsized": "2026-02-12T18:17:35.717178+0000", 2026-02-12T18:18:05.005 INFO:tasks.workunit.client.0.trial116.stdout: "mapping_epoch": 56, 2026-02-12T18:18:05.005 INFO:tasks.workunit.client.0.trial116.stdout: "log_start": "47'200", 2026-02-12T18:18:05.005 INFO:tasks.workunit.client.0.trial116.stdout: "ondisk_log_start": "47'200", 2026-02-12T18:18:05.005 INFO:tasks.workunit.client.0.trial116.stdout: "created": 41, 2026-02-12T18:18:05.005 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_clean": 42, 2026-02-12T18:18:05.005 INFO:tasks.workunit.client.0.trial116.stdout: "parent": "0.0", 2026-02-12T18:18:05.005 INFO:tasks.workunit.client.0.trial116.stdout: "parent_split_bits": 0, 2026-02-12T18:18:05.005 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub": "0'0", 2026-02-12T18:18:05.005 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub_stamp": "2026-02-12T18:17:29.908380+0000", 2026-02-12T18:18:05.005 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub": "0'0", 2026-02-12T18:18:05.005 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub_stamp": "2026-02-12T18:17:29.908380+0000", 2026-02-12T18:18:05.005 INFO:tasks.workunit.client.0.trial116.stdout: "last_clean_scrub_stamp": "2026-02-12T18:17:29.908380+0000", 2026-02-12T18:18:05.005 INFO:tasks.workunit.client.0.trial116.stdout: "objects_scrubbed": 0, 2026-02-12T18:18:05.005 INFO:tasks.workunit.client.0.trial116.stdout: "log_size": 101, 2026-02-12T18:18:05.005 INFO:tasks.workunit.client.0.trial116.stdout: "log_dups_size": 200, 2026-02-12T18:18:05.006 INFO:tasks.workunit.client.0.trial116.stdout: "ondisk_log_size": 101, 2026-02-12T18:18:05.006 INFO:tasks.workunit.client.0.trial116.stdout: "stats_invalid": false, 2026-02-12T18:18:05.006 INFO:tasks.workunit.client.0.trial116.stdout: "dirty_stats_invalid": false, 2026-02-12T18:18:05.006 INFO:tasks.workunit.client.0.trial116.stdout: "omap_stats_invalid": false, 2026-02-12T18:18:05.006 INFO:tasks.workunit.client.0.trial116.stdout: "hitset_stats_invalid": false, 2026-02-12T18:18:05.006 INFO:tasks.workunit.client.0.trial116.stdout: "hitset_bytes_stats_invalid": false, 2026-02-12T18:18:05.006 INFO:tasks.workunit.client.0.trial116.stdout: "pin_stats_invalid": false, 2026-02-12T18:18:05.006 INFO:tasks.workunit.client.0.trial116.stdout: "manifest_stats_invalid": false, 2026-02-12T18:18:05.006 INFO:tasks.workunit.client.0.trial116.stdout: "snaptrimq_len": 0, 2026-02-12T18:18:05.006 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub_duration": 0, 2026-02-12T18:18:05.006 INFO:tasks.workunit.client.0.trial116.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-13T21:41:30.320669+0000", 2026-02-12T18:18:05.006 INFO:tasks.workunit.client.0.trial116.stdout: "scrub_duration": 0, 2026-02-12T18:18:05.006 INFO:tasks.workunit.client.0.trial116.stdout: "objects_trimmed": 0, 2026-02-12T18:18:05.006 INFO:tasks.workunit.client.0.trial116.stdout: "snaptrim_duration": 0, 2026-02-12T18:18:05.006 INFO:tasks.workunit.client.0.trial116.stdout: "stat_sum": { 2026-02-12T18:18:05.006 INFO:tasks.workunit.client.0.trial116.stdout: "num_bytes": 1232896, 2026-02-12T18:18:05.006 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects": 301, 2026-02-12T18:18:05.006 INFO:tasks.workunit.client.0.trial116.stdout: "num_object_clones": 0, 2026-02-12T18:18:05.007 INFO:tasks.workunit.client.0.trial116.stdout: "num_object_copies": 1505, 2026-02-12T18:18:05.007 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_missing_on_primary": 0, 2026-02-12T18:18:05.007 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_missing": 0, 2026-02-12T18:18:05.007 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_degraded": 301, 2026-02-12T18:18:05.007 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_misplaced": 0, 2026-02-12T18:18:05.007 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_unfound": 0, 2026-02-12T18:18:05.007 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_dirty": 301, 2026-02-12T18:18:05.007 INFO:tasks.workunit.client.0.trial116.stdout: "num_whiteouts": 0, 2026-02-12T18:18:05.007 INFO:tasks.workunit.client.0.trial116.stdout: "num_read": 0, 2026-02-12T18:18:05.007 INFO:tasks.workunit.client.0.trial116.stdout: "num_read_kb": 0, 2026-02-12T18:18:05.007 INFO:tasks.workunit.client.0.trial116.stdout: "num_write": 301, 2026-02-12T18:18:05.007 INFO:tasks.workunit.client.0.trial116.stdout: "num_write_kb": 1204, 2026-02-12T18:18:05.007 INFO:tasks.workunit.client.0.trial116.stdout: "num_scrub_errors": 0, 2026-02-12T18:18:05.007 INFO:tasks.workunit.client.0.trial116.stdout: "num_shallow_scrub_errors": 0, 2026-02-12T18:18:05.007 INFO:tasks.workunit.client.0.trial116.stdout: "num_deep_scrub_errors": 0, 2026-02-12T18:18:05.007 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_recovered": 0, 2026-02-12T18:18:05.007 INFO:tasks.workunit.client.0.trial116.stdout: "num_bytes_recovered": 0, 2026-02-12T18:18:05.007 INFO:tasks.workunit.client.0.trial116.stdout: "num_keys_recovered": 0, 2026-02-12T18:18:05.007 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_omap": 0, 2026-02-12T18:18:05.008 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_hit_set_archive": 0, 2026-02-12T18:18:05.008 INFO:tasks.workunit.client.0.trial116.stdout: "num_bytes_hit_set_archive": 0, 2026-02-12T18:18:05.008 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush": 0, 2026-02-12T18:18:05.008 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush_kb": 0, 2026-02-12T18:18:05.008 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict": 0, 2026-02-12T18:18:05.008 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict_kb": 0, 2026-02-12T18:18:05.008 INFO:tasks.workunit.client.0.trial116.stdout: "num_promote": 0, 2026-02-12T18:18:05.008 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush_mode_high": 0, 2026-02-12T18:18:05.008 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush_mode_low": 0, 2026-02-12T18:18:05.008 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict_mode_some": 0, 2026-02-12T18:18:05.008 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict_mode_full": 0, 2026-02-12T18:18:05.008 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_pinned": 0, 2026-02-12T18:18:05.008 INFO:tasks.workunit.client.0.trial116.stdout: "num_legacy_snapsets": 0, 2026-02-12T18:18:05.008 INFO:tasks.workunit.client.0.trial116.stdout: "num_large_omap_objects": 0, 2026-02-12T18:18:05.008 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_manifest": 0, 2026-02-12T18:18:05.008 INFO:tasks.workunit.client.0.trial116.stdout: "num_omap_bytes": 0, 2026-02-12T18:18:05.008 INFO:tasks.workunit.client.0.trial116.stdout: "num_omap_keys": 0, 2026-02-12T18:18:05.008 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_repaired": 0 2026-02-12T18:18:05.008 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:18:05.009 INFO:tasks.workunit.client.0.trial116.stdout: "up": [ 2026-02-12T18:18:05.009 INFO:tasks.workunit.client.0.trial116.stdout: 3, 2026-02-12T18:18:05.009 INFO:tasks.workunit.client.0.trial116.stdout: 1, 2026-02-12T18:18:05.009 INFO:tasks.workunit.client.0.trial116.stdout: 4, 2026-02-12T18:18:05.009 INFO:tasks.workunit.client.0.trial116.stdout: 0, 2026-02-12T18:18:05.009 INFO:tasks.workunit.client.0.trial116.stdout: 2147483647 2026-02-12T18:18:05.009 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:18:05.009 INFO:tasks.workunit.client.0.trial116.stdout: "acting": [ 2026-02-12T18:18:05.009 INFO:tasks.workunit.client.0.trial116.stdout: 3, 2026-02-12T18:18:05.009 INFO:tasks.workunit.client.0.trial116.stdout: 1, 2026-02-12T18:18:05.009 INFO:tasks.workunit.client.0.trial116.stdout: 4, 2026-02-12T18:18:05.009 INFO:tasks.workunit.client.0.trial116.stdout: 0, 2026-02-12T18:18:05.009 INFO:tasks.workunit.client.0.trial116.stdout: 2147483647 2026-02-12T18:18:05.009 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:18:05.009 INFO:tasks.workunit.client.0.trial116.stdout: "avail_no_missing": [ 2026-02-12T18:18:05.009 INFO:tasks.workunit.client.0.trial116.stdout: "3(0)", 2026-02-12T18:18:05.009 INFO:tasks.workunit.client.0.trial116.stdout: "0(3)", 2026-02-12T18:18:05.009 INFO:tasks.workunit.client.0.trial116.stdout: "1(1)", 2026-02-12T18:18:05.009 INFO:tasks.workunit.client.0.trial116.stdout: "4(2)" 2026-02-12T18:18:05.009 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:18:05.010 INFO:tasks.workunit.client.0.trial116.stdout: "object_location_counts": [ 2026-02-12T18:18:05.010 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:18:05.010 INFO:tasks.workunit.client.0.trial116.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-12T18:18:05.010 INFO:tasks.workunit.client.0.trial116.stdout: "objects": 301 2026-02-12T18:18:05.010 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:18:05.010 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:18:05.010 INFO:tasks.workunit.client.0.trial116.stdout: "blocked_by": [], 2026-02-12T18:18:05.010 INFO:tasks.workunit.client.0.trial116.stdout: "up_primary": 3, 2026-02-12T18:18:05.010 INFO:tasks.workunit.client.0.trial116.stdout: "acting_primary": 3, 2026-02-12T18:18:05.010 INFO:tasks.workunit.client.0.trial116.stdout: "purged_snaps": [] 2026-02-12T18:18:05.010 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:18:05.010 INFO:tasks.workunit.client.0.trial116.stdout: "empty": 0, 2026-02-12T18:18:05.010 INFO:tasks.workunit.client.0.trial116.stdout: "dne": 0, 2026-02-12T18:18:05.010 INFO:tasks.workunit.client.0.trial116.stdout: "incomplete": 0, 2026-02-12T18:18:05.010 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_started": 57, 2026-02-12T18:18:05.010 INFO:tasks.workunit.client.0.trial116.stdout: "hit_set_history": { 2026-02-12T18:18:05.010 INFO:tasks.workunit.client.0.trial116.stdout: "current_last_update": "0'0", 2026-02-12T18:18:05.010 INFO:tasks.workunit.client.0.trial116.stdout: "history": [] 2026-02-12T18:18:05.010 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:18:05.010 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:18:05.011 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:18:05.011 INFO:tasks.workunit.client.0.trial116.stdout: "recovery_state": [ 2026-02-12T18:18:05.011 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:18:05.011 INFO:tasks.workunit.client.0.trial116.stdout: "name": "Started/Primary/Active", 2026-02-12T18:18:05.011 INFO:tasks.workunit.client.0.trial116.stdout: "enter_time": "2026-02-12T18:17:59.660806+0000", 2026-02-12T18:18:05.011 INFO:tasks.workunit.client.0.trial116.stdout: "might_have_unfound": [ 2026-02-12T18:18:05.011 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:18:05.011 INFO:tasks.workunit.client.0.trial116.stdout: "osd": "0(3)", 2026-02-12T18:18:05.011 INFO:tasks.workunit.client.0.trial116.stdout: "status": "already probed" 2026-02-12T18:18:05.011 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:18:05.011 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:18:05.011 INFO:tasks.workunit.client.0.trial116.stdout: "osd": "1(1)", 2026-02-12T18:18:05.011 INFO:tasks.workunit.client.0.trial116.stdout: "status": "already probed" 2026-02-12T18:18:05.011 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:18:05.011 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:18:05.011 INFO:tasks.workunit.client.0.trial116.stdout: "osd": "2(4)", 2026-02-12T18:18:05.011 INFO:tasks.workunit.client.0.trial116.stdout: "status": "osd is down" 2026-02-12T18:18:05.011 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:18:05.011 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:18:05.011 INFO:tasks.workunit.client.0.trial116.stdout: "osd": "4(2)", 2026-02-12T18:18:05.011 INFO:tasks.workunit.client.0.trial116.stdout: "status": "already probed" 2026-02-12T18:18:05.011 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:18:05.012 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:18:05.012 INFO:tasks.workunit.client.0.trial116.stdout: "recovery_progress": { 2026-02-12T18:18:05.012 INFO:tasks.workunit.client.0.trial116.stdout: "backfill_targets": [], 2026-02-12T18:18:05.012 INFO:tasks.workunit.client.0.trial116.stdout: "waiting_on_backfill": [], 2026-02-12T18:18:05.012 INFO:tasks.workunit.client.0.trial116.stdout: "last_backfill_started": "MIN", 2026-02-12T18:18:05.012 INFO:tasks.workunit.client.0.trial116.stdout: "backfill_info": { 2026-02-12T18:18:05.012 INFO:tasks.workunit.client.0.trial116.stdout: "begin": "MIN", 2026-02-12T18:18:05.012 INFO:tasks.workunit.client.0.trial116.stdout: "end": "MIN", 2026-02-12T18:18:05.012 INFO:tasks.workunit.client.0.trial116.stdout: "objects": [] 2026-02-12T18:18:05.012 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:18:05.012 INFO:tasks.workunit.client.0.trial116.stdout: "peer_backfill_info": [], 2026-02-12T18:18:05.012 INFO:tasks.workunit.client.0.trial116.stdout: "backfills_in_flight": [], 2026-02-12T18:18:05.012 INFO:tasks.workunit.client.0.trial116.stdout: "recovering": [], 2026-02-12T18:18:05.012 INFO:tasks.workunit.client.0.trial116.stdout: "pg_backend": { 2026-02-12T18:18:05.012 INFO:tasks.workunit.client.0.trial116.stdout: "recovery_ops": [], 2026-02-12T18:18:05.012 INFO:tasks.workunit.client.0.trial116.stdout: "read_ops": [] 2026-02-12T18:18:05.012 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:18:05.012 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:18:05.012 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:18:05.012 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:18:05.013 INFO:tasks.workunit.client.0.trial116.stdout: "name": "Started", 2026-02-12T18:18:05.013 INFO:tasks.workunit.client.0.trial116.stdout: "enter_time": "2026-02-12T18:17:59.560671+0000" 2026-02-12T18:18:05.013 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:18:05.013 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:18:05.013 INFO:tasks.workunit.client.0.trial116.stdout: "scrubber": { 2026-02-12T18:18:05.013 INFO:tasks.workunit.client.0.trial116.stdout: "active": false, 2026-02-12T18:18:05.013 INFO:tasks.workunit.client.0.trial116.stdout: "must_scrub": false, 2026-02-12T18:18:05.013 INFO:tasks.workunit.client.0.trial116.stdout: "must_deep_scrub": false, 2026-02-12T18:18:05.013 INFO:tasks.workunit.client.0.trial116.stdout: "must_repair": false, 2026-02-12T18:18:05.013 INFO:tasks.workunit.client.0.trial116.stdout: "sched_time": "2026-02-13T18:30:13.577523+0000", 2026-02-12T18:18:05.013 INFO:tasks.workunit.client.0.trial116.stdout: "schedule": "not registered for scrubbing", 2026-02-12T18:18:05.013 INFO:tasks.workunit.client.0.trial116.stdout: "urgency": "periodic-regular", 2026-02-12T18:18:05.013 INFO:tasks.workunit.client.0.trial116.stdout: "shallow-target": { 2026-02-12T18:18:05.013 INFO:tasks.workunit.client.0.trial116.stdout: "pgid": "2.0s0", 2026-02-12T18:18:05.013 INFO:tasks.workunit.client.0.trial116.stdout: "level": "shallow", 2026-02-12T18:18:05.013 INFO:tasks.workunit.client.0.trial116.stdout: "urgency": "periodic-regular", 2026-02-12T18:18:05.013 INFO:tasks.workunit.client.0.trial116.stdout: "sched_time": "2026-02-13T18:30:13.577523+0000", 2026-02-12T18:18:05.013 INFO:tasks.workunit.client.0.trial116.stdout: "orig_sched_time": "2026-02-13T18:30:13.577523+0000", 2026-02-12T18:18:05.013 INFO:tasks.workunit.client.0.trial116.stdout: "last_issue": "ok", 2026-02-12T18:18:05.013 INFO:tasks.workunit.client.0.trial116.stdout: "forced": false, 2026-02-12T18:18:05.013 INFO:tasks.workunit.client.0.trial116.stdout: "eligible": false, 2026-02-12T18:18:05.014 INFO:tasks.workunit.client.0.trial116.stdout: "queued": false, 2026-02-12T18:18:05.014 INFO:tasks.workunit.client.0.trial116.stdout: "active": false 2026-02-12T18:18:05.014 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:18:05.014 INFO:tasks.workunit.client.0.trial116.stdout: "deep-target": { 2026-02-12T18:18:05.014 INFO:tasks.workunit.client.0.trial116.stdout: "pgid": "2.0s0", 2026-02-12T18:18:05.014 INFO:tasks.workunit.client.0.trial116.stdout: "level": "deep", 2026-02-12T18:18:05.014 INFO:tasks.workunit.client.0.trial116.stdout: "urgency": "periodic-regular", 2026-02-12T18:18:05.014 INFO:tasks.workunit.client.0.trial116.stdout: "sched_time": "2026-02-18T08:10:29.973769+0000", 2026-02-12T18:18:05.014 INFO:tasks.workunit.client.0.trial116.stdout: "orig_sched_time": "2026-02-18T08:10:29.973769+0000", 2026-02-12T18:18:05.014 INFO:tasks.workunit.client.0.trial116.stdout: "last_issue": "ok", 2026-02-12T18:18:05.014 INFO:tasks.workunit.client.0.trial116.stdout: "forced": false, 2026-02-12T18:18:05.014 INFO:tasks.workunit.client.0.trial116.stdout: "eligible": false, 2026-02-12T18:18:05.014 INFO:tasks.workunit.client.0.trial116.stdout: "queued": false, 2026-02-12T18:18:05.014 INFO:tasks.workunit.client.0.trial116.stdout: "active": false 2026-02-12T18:18:05.014 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:18:05.014 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:18:05.014 INFO:tasks.workunit.client.0.trial116.stdout: "agent_state": {} 2026-02-12T18:18:05.014 INFO:tasks.workunit.client.0.trial116.stdout:} 2026-02-12T18:18:05.015 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:590: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-12T18:18:05.015 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:590: TEST_ec_backfill_unfound: grep -q obj250 2026-02-12T18:18:05.033 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:592: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-12T18:18:05.033 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:592: TEST_ec_backfill_unfound: jq .available_might_have_unfound 2026-02-12T18:18:05.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:592: TEST_ec_backfill_unfound: check=true 2026-02-12T18:18:05.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: test true == true 2026-02-12T18:18:05.098 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-12T18:18:05.098 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: jq '.might_have_unfound[0].status' 2026-02-12T18:18:05.162 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: eval 'check="osd' is 'down"' 2026-02-12T18:18:05.162 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: check='osd is down' 2026-02-12T18:18:05.163 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:596: TEST_ec_backfill_unfound: test 'osd is down' == 'osd is down' 2026-02-12T18:18:05.163 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-12T18:18:05.163 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: jq '.might_have_unfound[0].osd' 2026-02-12T18:18:05.232 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: eval 'check="2(4)"' 2026-02-12T18:18:05.233 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: check='2(4)' 2026-02-12T18:18:05.233 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:599: TEST_ec_backfill_unfound: test '2(4)' == '2(4)' 2026-02-12T18:18:05.233 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:601: TEST_ec_backfill_unfound: activate_osd td/test-erasure-eio 2 2026-02-12T18:18:05.233 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-12T18:18:05.233 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T18:18:05.233 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-12T18:18:05.233 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T18:18:05.233 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-12T18:18:05.233 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:18:05.233 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:18:05.233 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:18:05.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:18:05.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-12T18:18:05.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-12T18:18:05.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T18:18:05.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T18:18:05.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:18:05.234 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T18:18:05.234 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:18:05.234 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:18:05.234 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:18:05.234 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:18:05.234 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:18:05.234 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:18:05.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:18:05.235 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:18:05.235 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:18:05.235 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:18:05.235 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:18:05.235 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:18:05.235 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:18:05.235 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:18:05.235 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T18:18:05.235 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-12T18:18:05.235 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-12T18:18:05.235 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-12T18:18:05.236 INFO:tasks.workunit.client.0.trial116.stdout:start osd.2 2026-02-12T18:18:05.236 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:18:05.236 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-12T18:18:05.237 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-12T18:18:05.237 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T18:18:05.237 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T18:18:05.237 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T18:18:05.246 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:05.242+0000 7fbc94e0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:05.247 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:05.244+0000 7fbc94e0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:05.248 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:05.244+0000 7fbc94e0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:05.440 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:05.437+0000 7fbc94e0f780 -1 Falling back to public interface 2026-02-12T18:18:05.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-12T18:18:05.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:18:05.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T18:18:05.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:18:05.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:18:05.473 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:18:05.473 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:18:05.473 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:18:05.474 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:18:05.474 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:18:05.698 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:18:05.794 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:05.791+0000 7fbc94e0f780 -1 osd.2 55 log_to_monitors true 2026-02-12T18:18:06.700 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:18:06.700 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:18:06.701 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:18:06.701 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:18:06.701 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:18:06.701 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:18:06.924 INFO:tasks.workunit.client.0.trial116.stdout:osd.2 up in weight 1 up_from 59 up_thru 20 down_at 56 last_clean_interval [49,55) [v2:127.0.0.1:6818/96999591,v1:127.0.0.1:6819/96999591] [v2:127.0.0.1:6820/96999591,v1:127.0.0.1:6821/96999591] exists,up 5de218c3-c7f2-4940-a6fd-2eb5d4aafaee 2026-02-12T18:18:06.925 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:18:06.925 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:18:06.925 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:18:06.925 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:604: TEST_ec_backfill_unfound: timeout 5 rados -p pool-jerasure get obj250 td/test-erasure-eio/CHECK 2026-02-12T18:18:11.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:605: TEST_ec_backfill_unfound: test 124 = 124 2026-02-12T18:18:11.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:607: TEST_ec_backfill_unfound: ceph pg 2.0 mark_unfound_lost delete 2026-02-12T18:18:11.994 INFO:tasks.workunit.client.0.trial116.stderr:pg has no unfound objects 2026-02-12T18:18:12.000 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:609: TEST_ec_backfill_unfound: wait_for_clean 2026-02-12T18:18:12.000 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:18:12.000 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:18:12.000 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:18:12.000 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:18:12.001 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:18:12.001 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:18:12.001 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:18:12.001 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:18:12.001 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:18:12.083 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:18:12.084 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:18:12.084 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:18:12.084 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:18:12.084 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:18:12.084 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:18:12.318 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:18:12.318 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:18:12.318 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:18:12.318 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:18:12.319 INFO:tasks.workunit.client.0.trial116.stderr:4' 2026-02-12T18:18:12.319 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:18:12.319 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:18:12.319 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:18:12.384 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803791 2026-02-12T18:18:12.384 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803791 2026-02-12T18:18:12.384 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791' 2026-02-12T18:18:12.384 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:18:12.384 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:18:12.447 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574863 2026-02-12T18:18:12.448 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574863 2026-02-12T18:18:12.448 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-55834574863' 2026-02-12T18:18:12.448 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:18:12.448 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:18:12.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=253403070468 2026-02-12T18:18:12.512 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 253403070468 2026-02-12T18:18:12.512 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-55834574863 2-253403070468' 2026-02-12T18:18:12.512 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:18:12.512 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:18:12.575 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964117007 2026-02-12T18:18:12.575 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964117007 2026-02-12T18:18:12.575 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-55834574863 2-253403070468 3-115964117007' 2026-02-12T18:18:12.575 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:18:12.575 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-12T18:18:12.641 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=154618822669 2026-02-12T18:18:12.641 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 154618822669 2026-02-12T18:18:12.641 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-55834574863 2-253403070468 3-115964117007 4-154618822669' 2026-02-12T18:18:12.642 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:18:12.642 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803791 2026-02-12T18:18:12.642 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:18:12.642 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:18:12.643 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803791 2026-02-12T18:18:12.643 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:18:12.644 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803791 2026-02-12T18:18:12.644 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803791' 2026-02-12T18:18:12.644 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.0 seq 25769803791 2026-02-12T18:18:12.644 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:18:12.871 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803789 -lt 25769803791 2026-02-12T18:18:12.871 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:18:13.872 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:18:13.872 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:18:14.099 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803791 -lt 25769803791 2026-02-12T18:18:14.099 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:18:14.099 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574863 2026-02-12T18:18:14.100 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:18:14.100 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:18:14.101 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574863 2026-02-12T18:18:14.101 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:18:14.102 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574863 2026-02-12T18:18:14.102 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574863' 2026-02-12T18:18:14.102 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.1 seq 55834574863 2026-02-12T18:18:14.102 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:18:14.330 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574863 -lt 55834574863 2026-02-12T18:18:14.331 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:18:14.331 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-253403070468 2026-02-12T18:18:14.331 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:18:14.332 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:18:14.332 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-253403070468 2026-02-12T18:18:14.333 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:18:14.333 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=253403070468 2026-02-12T18:18:14.334 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 253403070468' 2026-02-12T18:18:14.334 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.2 seq 253403070468 2026-02-12T18:18:14.334 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:18:14.562 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 253403070468 -lt 253403070468 2026-02-12T18:18:14.562 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:18:14.563 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-115964117007 2026-02-12T18:18:14.563 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:18:14.563 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:18:14.564 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964117007 2026-02-12T18:18:14.564 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:18:14.565 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964117007 2026-02-12T18:18:14.566 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 115964117007' 2026-02-12T18:18:14.566 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.3 seq 115964117007 2026-02-12T18:18:14.566 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:18:14.791 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964117007 -lt 115964117007 2026-02-12T18:18:14.791 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:18:14.792 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-154618822669 2026-02-12T18:18:14.792 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:18:14.793 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-12T18:18:14.793 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-154618822669 2026-02-12T18:18:14.793 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:18:14.794 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=154618822669 2026-02-12T18:18:14.794 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.4 seq 154618822669 2026-02-12T18:18:14.795 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 154618822669' 2026-02-12T18:18:14.795 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-12T18:18:15.018 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 154618822669 -lt 154618822669 2026-02-12T18:18:15.018 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:18:15.019 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:18:15.019 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:18:15.330 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:18:15.331 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:18:15.331 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:18:15.331 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:18:15.331 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:18:15.331 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:18:15.331 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:18:15.331 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:18:15.556 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:18:15.556 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:18:15.557 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:18:15.557 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:18:15.882 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:18:15.882 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:18:15.882 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:18:15.882 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: seq 1 300 2026-02-12T18:18:15.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:15.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj1 = obj250 ']' 2026-02-12T18:18:15.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj1 td/test-erasure-eio/CHECK 2026-02-12T18:18:15.899 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:15.900 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:15.900 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj2 = obj250 ']' 2026-02-12T18:18:15.901 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj2 td/test-erasure-eio/CHECK 2026-02-12T18:18:15.916 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:15.917 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:15.917 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj3 = obj250 ']' 2026-02-12T18:18:15.917 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj3 td/test-erasure-eio/CHECK 2026-02-12T18:18:15.934 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:15.935 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:15.935 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj4 = obj250 ']' 2026-02-12T18:18:15.935 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj4 td/test-erasure-eio/CHECK 2026-02-12T18:18:15.951 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:15.952 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:15.952 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj5 = obj250 ']' 2026-02-12T18:18:15.952 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj5 td/test-erasure-eio/CHECK 2026-02-12T18:18:15.967 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:15.967 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:15.968 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj6 = obj250 ']' 2026-02-12T18:18:15.968 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj6 td/test-erasure-eio/CHECK 2026-02-12T18:18:15.983 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:15.984 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:15.984 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj7 = obj250 ']' 2026-02-12T18:18:15.984 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj7 td/test-erasure-eio/CHECK 2026-02-12T18:18:15.999 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.000 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.000 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj8 = obj250 ']' 2026-02-12T18:18:16.000 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj8 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.016 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.017 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.017 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj9 = obj250 ']' 2026-02-12T18:18:16.017 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj9 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.032 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.033 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.033 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj10 = obj250 ']' 2026-02-12T18:18:16.033 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj10 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.048 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.049 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.050 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj11 = obj250 ']' 2026-02-12T18:18:16.050 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj11 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.065 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.066 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.066 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj12 = obj250 ']' 2026-02-12T18:18:16.067 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj12 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.082 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.083 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.083 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj13 = obj250 ']' 2026-02-12T18:18:16.083 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj13 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.098 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.099 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.099 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj14 = obj250 ']' 2026-02-12T18:18:16.099 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj14 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.114 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.115 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.115 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj15 = obj250 ']' 2026-02-12T18:18:16.116 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj15 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.131 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.132 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.132 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj16 = obj250 ']' 2026-02-12T18:18:16.132 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj16 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.147 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.148 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.148 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj17 = obj250 ']' 2026-02-12T18:18:16.148 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj17 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.163 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.164 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.164 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj18 = obj250 ']' 2026-02-12T18:18:16.164 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj18 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.179 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.180 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.181 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj19 = obj250 ']' 2026-02-12T18:18:16.181 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj19 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.196 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.197 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.197 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj20 = obj250 ']' 2026-02-12T18:18:16.198 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj20 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.213 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.214 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.214 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj21 = obj250 ']' 2026-02-12T18:18:16.214 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj21 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.230 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.231 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.231 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj22 = obj250 ']' 2026-02-12T18:18:16.231 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj22 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.246 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.247 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.247 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj23 = obj250 ']' 2026-02-12T18:18:16.248 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj23 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.264 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.264 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj24 = obj250 ']' 2026-02-12T18:18:16.264 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj24 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.281 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.282 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.282 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj25 = obj250 ']' 2026-02-12T18:18:16.282 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj25 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.298 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj26 = obj250 ']' 2026-02-12T18:18:16.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj26 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.315 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.316 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj27 = obj250 ']' 2026-02-12T18:18:16.316 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj27 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.331 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.332 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.332 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj28 = obj250 ']' 2026-02-12T18:18:16.333 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj28 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.348 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj29 = obj250 ']' 2026-02-12T18:18:16.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj29 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.365 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.366 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.366 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj30 = obj250 ']' 2026-02-12T18:18:16.366 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj30 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.382 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.383 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.383 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj31 = obj250 ']' 2026-02-12T18:18:16.383 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj31 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.399 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.400 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.400 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj32 = obj250 ']' 2026-02-12T18:18:16.400 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj32 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.415 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.416 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.416 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj33 = obj250 ']' 2026-02-12T18:18:16.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj33 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.432 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj34 = obj250 ']' 2026-02-12T18:18:16.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj34 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.449 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.450 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.450 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj35 = obj250 ']' 2026-02-12T18:18:16.450 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj35 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.466 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.467 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.467 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj36 = obj250 ']' 2026-02-12T18:18:16.467 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj36 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.483 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.484 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.484 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj37 = obj250 ']' 2026-02-12T18:18:16.484 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj37 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.500 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.500 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.501 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj38 = obj250 ']' 2026-02-12T18:18:16.501 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj38 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.517 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.518 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.518 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj39 = obj250 ']' 2026-02-12T18:18:16.518 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj39 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.533 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.534 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.534 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj40 = obj250 ']' 2026-02-12T18:18:16.534 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj40 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.550 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.551 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.551 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj41 = obj250 ']' 2026-02-12T18:18:16.552 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj41 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.567 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.568 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.568 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj42 = obj250 ']' 2026-02-12T18:18:16.568 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj42 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.583 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.584 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.584 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj43 = obj250 ']' 2026-02-12T18:18:16.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj43 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.600 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.601 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.601 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj44 = obj250 ']' 2026-02-12T18:18:16.601 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj44 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.617 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.618 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.618 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj45 = obj250 ']' 2026-02-12T18:18:16.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj45 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.634 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj46 = obj250 ']' 2026-02-12T18:18:16.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj46 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.652 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.652 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj47 = obj250 ']' 2026-02-12T18:18:16.652 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj47 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.668 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.669 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.669 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj48 = obj250 ']' 2026-02-12T18:18:16.669 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj48 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.687 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.687 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj49 = obj250 ']' 2026-02-12T18:18:16.687 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj49 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.703 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.704 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.704 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj50 = obj250 ']' 2026-02-12T18:18:16.704 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj50 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.720 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.720 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj51 = obj250 ']' 2026-02-12T18:18:16.720 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj51 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.736 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.736 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.737 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj52 = obj250 ']' 2026-02-12T18:18:16.737 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj52 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.752 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.754 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.754 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj53 = obj250 ']' 2026-02-12T18:18:16.754 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj53 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.769 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.770 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.770 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj54 = obj250 ']' 2026-02-12T18:18:16.770 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj54 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.787 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.788 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.788 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj55 = obj250 ']' 2026-02-12T18:18:16.788 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj55 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.803 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.804 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.805 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj56 = obj250 ']' 2026-02-12T18:18:16.805 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj56 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.821 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.822 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.822 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj57 = obj250 ']' 2026-02-12T18:18:16.822 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj57 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.838 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.839 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.839 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj58 = obj250 ']' 2026-02-12T18:18:16.839 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj58 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.856 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj59 = obj250 ']' 2026-02-12T18:18:16.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj59 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.872 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.873 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.874 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj60 = obj250 ']' 2026-02-12T18:18:16.874 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj60 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.889 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.890 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.890 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj61 = obj250 ']' 2026-02-12T18:18:16.890 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj61 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.907 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.907 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj62 = obj250 ']' 2026-02-12T18:18:16.907 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj62 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.922 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.923 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.923 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj63 = obj250 ']' 2026-02-12T18:18:16.924 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj63 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.939 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.941 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj64 = obj250 ']' 2026-02-12T18:18:16.941 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj64 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.956 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.957 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.957 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj65 = obj250 ']' 2026-02-12T18:18:16.957 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj65 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.972 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.973 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.973 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj66 = obj250 ']' 2026-02-12T18:18:16.973 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj66 td/test-erasure-eio/CHECK 2026-02-12T18:18:16.989 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:16.990 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:16.990 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj67 = obj250 ']' 2026-02-12T18:18:16.990 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj67 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.006 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj68 = obj250 ']' 2026-02-12T18:18:17.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj68 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.023 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.024 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.024 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj69 = obj250 ']' 2026-02-12T18:18:17.024 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj69 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.039 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.040 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.040 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj70 = obj250 ']' 2026-02-12T18:18:17.040 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj70 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.056 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.057 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.057 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj71 = obj250 ']' 2026-02-12T18:18:17.057 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj71 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.073 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.074 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.074 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj72 = obj250 ']' 2026-02-12T18:18:17.074 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj72 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.089 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj73 = obj250 ']' 2026-02-12T18:18:17.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj73 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.106 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.106 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.106 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj74 = obj250 ']' 2026-02-12T18:18:17.106 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj74 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.123 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.124 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.124 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj75 = obj250 ']' 2026-02-12T18:18:17.124 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj75 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.140 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.141 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.141 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj76 = obj250 ']' 2026-02-12T18:18:17.141 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj76 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.157 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj77 = obj250 ']' 2026-02-12T18:18:17.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj77 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.175 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.176 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.176 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj78 = obj250 ']' 2026-02-12T18:18:17.176 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj78 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.192 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.193 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.193 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj79 = obj250 ']' 2026-02-12T18:18:17.193 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj79 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.208 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.209 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.209 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj80 = obj250 ']' 2026-02-12T18:18:17.209 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj80 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.226 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.226 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj81 = obj250 ']' 2026-02-12T18:18:17.226 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj81 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.243 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj82 = obj250 ']' 2026-02-12T18:18:17.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj82 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.260 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj83 = obj250 ']' 2026-02-12T18:18:17.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj83 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.278 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.278 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj84 = obj250 ']' 2026-02-12T18:18:17.278 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj84 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.294 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.295 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.295 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj85 = obj250 ']' 2026-02-12T18:18:17.295 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj85 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.310 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.311 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.311 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj86 = obj250 ']' 2026-02-12T18:18:17.311 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj86 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.329 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.329 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj87 = obj250 ']' 2026-02-12T18:18:17.329 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj87 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.345 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.345 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj88 = obj250 ']' 2026-02-12T18:18:17.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj88 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.361 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.362 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.362 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj89 = obj250 ']' 2026-02-12T18:18:17.363 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj89 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.380 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.380 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj90 = obj250 ']' 2026-02-12T18:18:17.380 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj90 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.396 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.397 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.397 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj91 = obj250 ']' 2026-02-12T18:18:17.397 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj91 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.413 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.414 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.414 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj92 = obj250 ']' 2026-02-12T18:18:17.414 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj92 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj93 = obj250 ']' 2026-02-12T18:18:17.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj93 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.447 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.448 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.448 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj94 = obj250 ']' 2026-02-12T18:18:17.448 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj94 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.463 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.464 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.465 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj95 = obj250 ']' 2026-02-12T18:18:17.465 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj95 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.480 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.481 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.481 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj96 = obj250 ']' 2026-02-12T18:18:17.481 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj96 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.497 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.498 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.498 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj97 = obj250 ']' 2026-02-12T18:18:17.498 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj97 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.514 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.515 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.515 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj98 = obj250 ']' 2026-02-12T18:18:17.515 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj98 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.531 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.532 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.532 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj99 = obj250 ']' 2026-02-12T18:18:17.532 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj99 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.547 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.548 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.548 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj100 = obj250 ']' 2026-02-12T18:18:17.549 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj100 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.565 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.566 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.567 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj101 = obj250 ']' 2026-02-12T18:18:17.567 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj101 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.583 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.584 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.584 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj102 = obj250 ']' 2026-02-12T18:18:17.584 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj102 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.599 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.600 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.600 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj103 = obj250 ']' 2026-02-12T18:18:17.600 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj103 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.617 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.618 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.618 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj104 = obj250 ']' 2026-02-12T18:18:17.618 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj104 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.634 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj105 = obj250 ']' 2026-02-12T18:18:17.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj105 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.652 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.652 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj106 = obj250 ']' 2026-02-12T18:18:17.652 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj106 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.667 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.668 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.669 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj107 = obj250 ']' 2026-02-12T18:18:17.669 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj107 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.684 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.685 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.685 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj108 = obj250 ']' 2026-02-12T18:18:17.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj108 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.701 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.702 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.702 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj109 = obj250 ']' 2026-02-12T18:18:17.702 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj109 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.718 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj110 = obj250 ']' 2026-02-12T18:18:17.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj110 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.734 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.735 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.735 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj111 = obj250 ']' 2026-02-12T18:18:17.735 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj111 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.751 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.752 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.752 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj112 = obj250 ']' 2026-02-12T18:18:17.752 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj112 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.767 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.769 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj113 = obj250 ']' 2026-02-12T18:18:17.769 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj113 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.785 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.786 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.786 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj114 = obj250 ']' 2026-02-12T18:18:17.786 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj114 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj115 = obj250 ']' 2026-02-12T18:18:17.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj115 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.817 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.818 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.818 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj116 = obj250 ']' 2026-02-12T18:18:17.819 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj116 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.835 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.836 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.836 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj117 = obj250 ']' 2026-02-12T18:18:17.836 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj117 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.852 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.853 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.853 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj118 = obj250 ']' 2026-02-12T18:18:17.853 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj118 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.868 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.869 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.870 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj119 = obj250 ']' 2026-02-12T18:18:17.870 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj119 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.885 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.886 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.886 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj120 = obj250 ']' 2026-02-12T18:18:17.886 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj120 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj121 = obj250 ']' 2026-02-12T18:18:17.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj121 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.919 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.920 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.920 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj122 = obj250 ']' 2026-02-12T18:18:17.921 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj122 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.936 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.937 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.937 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj123 = obj250 ']' 2026-02-12T18:18:17.937 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj123 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.953 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.954 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.954 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj124 = obj250 ']' 2026-02-12T18:18:17.954 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj124 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.970 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.971 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.971 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj125 = obj250 ']' 2026-02-12T18:18:17.971 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj125 td/test-erasure-eio/CHECK 2026-02-12T18:18:17.987 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:17.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:17.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj126 = obj250 ']' 2026-02-12T18:18:17.989 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj126 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.005 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.005 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj127 = obj250 ']' 2026-02-12T18:18:18.005 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj127 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.021 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.022 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.022 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj128 = obj250 ']' 2026-02-12T18:18:18.022 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj128 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.038 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.038 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.039 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj129 = obj250 ']' 2026-02-12T18:18:18.039 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj129 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.054 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.055 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.055 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj130 = obj250 ']' 2026-02-12T18:18:18.055 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj130 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.071 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.072 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.072 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj131 = obj250 ']' 2026-02-12T18:18:18.072 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj131 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.087 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.088 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.089 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj132 = obj250 ']' 2026-02-12T18:18:18.089 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj132 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.104 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.105 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.105 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj133 = obj250 ']' 2026-02-12T18:18:18.105 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj133 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.121 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj134 = obj250 ']' 2026-02-12T18:18:18.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj134 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.137 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.138 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.138 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj135 = obj250 ']' 2026-02-12T18:18:18.138 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj135 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.154 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.155 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.155 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj136 = obj250 ']' 2026-02-12T18:18:18.155 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj136 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.171 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.172 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.172 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj137 = obj250 ']' 2026-02-12T18:18:18.172 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj137 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.187 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.188 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.188 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj138 = obj250 ']' 2026-02-12T18:18:18.188 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj138 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.204 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.205 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.205 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj139 = obj250 ']' 2026-02-12T18:18:18.205 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj139 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.220 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.221 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.221 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj140 = obj250 ']' 2026-02-12T18:18:18.221 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj140 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.236 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.237 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.237 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj141 = obj250 ']' 2026-02-12T18:18:18.237 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj141 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.252 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.253 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.253 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj142 = obj250 ']' 2026-02-12T18:18:18.253 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj142 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.268 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.269 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.270 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj143 = obj250 ']' 2026-02-12T18:18:18.270 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj143 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.285 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.286 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.286 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj144 = obj250 ']' 2026-02-12T18:18:18.286 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj144 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.302 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.303 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.303 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj145 = obj250 ']' 2026-02-12T18:18:18.303 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj145 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.319 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj146 = obj250 ']' 2026-02-12T18:18:18.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj146 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.337 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.338 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.338 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj147 = obj250 ']' 2026-02-12T18:18:18.338 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj147 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.353 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.354 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.354 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj148 = obj250 ']' 2026-02-12T18:18:18.354 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj148 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.370 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.371 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.371 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj149 = obj250 ']' 2026-02-12T18:18:18.371 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj149 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.388 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.388 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj150 = obj250 ']' 2026-02-12T18:18:18.388 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj150 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.403 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.404 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.404 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj151 = obj250 ']' 2026-02-12T18:18:18.404 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj151 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.421 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.422 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.422 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj152 = obj250 ']' 2026-02-12T18:18:18.422 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj152 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.437 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.438 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.438 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj153 = obj250 ']' 2026-02-12T18:18:18.438 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj153 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.453 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.454 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj154 = obj250 ']' 2026-02-12T18:18:18.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj154 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.470 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.471 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj155 = obj250 ']' 2026-02-12T18:18:18.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj155 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.486 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.487 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.487 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj156 = obj250 ']' 2026-02-12T18:18:18.487 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj156 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.502 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.503 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.503 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj157 = obj250 ']' 2026-02-12T18:18:18.504 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj157 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.519 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.521 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.521 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj158 = obj250 ']' 2026-02-12T18:18:18.521 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj158 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.536 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.537 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.537 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj159 = obj250 ']' 2026-02-12T18:18:18.538 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj159 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.553 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.554 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.555 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj160 = obj250 ']' 2026-02-12T18:18:18.555 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj160 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.571 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.572 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.572 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj161 = obj250 ']' 2026-02-12T18:18:18.572 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj161 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.588 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj162 = obj250 ']' 2026-02-12T18:18:18.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj162 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.604 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.605 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.605 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj163 = obj250 ']' 2026-02-12T18:18:18.605 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj163 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj164 = obj250 ']' 2026-02-12T18:18:18.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj164 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.637 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj165 = obj250 ']' 2026-02-12T18:18:18.639 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj165 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.654 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.655 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.655 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj166 = obj250 ']' 2026-02-12T18:18:18.655 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj166 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.671 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.672 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.672 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj167 = obj250 ']' 2026-02-12T18:18:18.672 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj167 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.688 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.689 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.689 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj168 = obj250 ']' 2026-02-12T18:18:18.689 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj168 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.705 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj169 = obj250 ']' 2026-02-12T18:18:18.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj169 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.721 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.722 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.723 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj170 = obj250 ']' 2026-02-12T18:18:18.723 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj170 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.738 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.739 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.739 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj171 = obj250 ']' 2026-02-12T18:18:18.739 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj171 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.755 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.756 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.756 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj172 = obj250 ']' 2026-02-12T18:18:18.756 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj172 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.772 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.773 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.773 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj173 = obj250 ']' 2026-02-12T18:18:18.773 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj173 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.788 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.789 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.789 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj174 = obj250 ']' 2026-02-12T18:18:18.789 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj174 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.805 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.806 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.806 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj175 = obj250 ']' 2026-02-12T18:18:18.806 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj175 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.822 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.822 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.823 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj176 = obj250 ']' 2026-02-12T18:18:18.823 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj176 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.838 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.839 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.839 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj177 = obj250 ']' 2026-02-12T18:18:18.839 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj177 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.854 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.855 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.855 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj178 = obj250 ']' 2026-02-12T18:18:18.855 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj178 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.871 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.872 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.872 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj179 = obj250 ']' 2026-02-12T18:18:18.872 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj179 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.887 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.888 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.888 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj180 = obj250 ']' 2026-02-12T18:18:18.889 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj180 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.905 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.905 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj181 = obj250 ']' 2026-02-12T18:18:18.905 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj181 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.921 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.922 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.922 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj182 = obj250 ']' 2026-02-12T18:18:18.922 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj182 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.938 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.939 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.939 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj183 = obj250 ']' 2026-02-12T18:18:18.939 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj183 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.955 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.956 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.956 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj184 = obj250 ']' 2026-02-12T18:18:18.956 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj184 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.971 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.972 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.972 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj185 = obj250 ']' 2026-02-12T18:18:18.972 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj185 td/test-erasure-eio/CHECK 2026-02-12T18:18:18.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:18.989 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:18.989 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj186 = obj250 ']' 2026-02-12T18:18:18.989 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj186 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.005 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.005 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj187 = obj250 ']' 2026-02-12T18:18:19.005 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj187 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.021 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.022 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.022 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj188 = obj250 ']' 2026-02-12T18:18:19.022 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj188 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.037 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.038 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.039 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj189 = obj250 ']' 2026-02-12T18:18:19.039 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj189 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.055 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.056 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.056 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj190 = obj250 ']' 2026-02-12T18:18:19.056 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj190 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.072 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.073 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.073 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj191 = obj250 ']' 2026-02-12T18:18:19.073 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj191 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.089 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj192 = obj250 ']' 2026-02-12T18:18:19.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj192 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.106 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.107 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.107 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj193 = obj250 ']' 2026-02-12T18:18:19.107 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj193 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.123 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.124 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.124 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj194 = obj250 ']' 2026-02-12T18:18:19.124 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj194 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.140 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.141 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.141 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj195 = obj250 ']' 2026-02-12T18:18:19.141 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj195 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.157 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj196 = obj250 ']' 2026-02-12T18:18:19.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj196 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.174 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.175 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.175 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj197 = obj250 ']' 2026-02-12T18:18:19.175 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj197 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.190 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.191 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.191 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj198 = obj250 ']' 2026-02-12T18:18:19.192 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj198 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.207 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.208 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.208 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj199 = obj250 ']' 2026-02-12T18:18:19.208 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj199 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.224 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj200 = obj250 ']' 2026-02-12T18:18:19.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj200 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.241 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.242 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.242 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj201 = obj250 ']' 2026-02-12T18:18:19.242 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj201 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.258 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.259 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.259 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj202 = obj250 ']' 2026-02-12T18:18:19.259 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj202 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.274 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj203 = obj250 ']' 2026-02-12T18:18:19.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj203 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.292 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.292 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj204 = obj250 ']' 2026-02-12T18:18:19.292 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj204 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.307 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.308 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.309 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj205 = obj250 ']' 2026-02-12T18:18:19.309 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj205 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.324 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.325 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.325 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj206 = obj250 ']' 2026-02-12T18:18:19.325 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj206 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.340 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.341 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.342 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj207 = obj250 ']' 2026-02-12T18:18:19.342 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj207 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.357 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.358 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.358 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj208 = obj250 ']' 2026-02-12T18:18:19.358 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj208 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.374 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.375 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.375 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj209 = obj250 ']' 2026-02-12T18:18:19.375 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj209 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.391 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.392 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.393 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj210 = obj250 ']' 2026-02-12T18:18:19.393 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj210 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj211 = obj250 ']' 2026-02-12T18:18:19.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj211 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.425 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj212 = obj250 ']' 2026-02-12T18:18:19.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj212 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.441 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj213 = obj250 ']' 2026-02-12T18:18:19.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj213 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.458 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.459 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.459 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj214 = obj250 ']' 2026-02-12T18:18:19.459 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj214 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.474 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.475 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.476 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj215 = obj250 ']' 2026-02-12T18:18:19.476 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj215 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.491 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.492 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.492 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj216 = obj250 ']' 2026-02-12T18:18:19.492 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj216 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.509 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.509 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj217 = obj250 ']' 2026-02-12T18:18:19.509 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj217 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.525 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.526 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.526 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj218 = obj250 ']' 2026-02-12T18:18:19.526 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj218 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.542 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj219 = obj250 ']' 2026-02-12T18:18:19.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj219 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.560 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.560 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.561 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj220 = obj250 ']' 2026-02-12T18:18:19.561 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj220 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.576 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.577 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.577 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj221 = obj250 ']' 2026-02-12T18:18:19.577 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj221 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.593 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.594 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.594 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj222 = obj250 ']' 2026-02-12T18:18:19.594 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj222 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.610 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.611 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.611 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj223 = obj250 ']' 2026-02-12T18:18:19.611 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj223 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj224 = obj250 ']' 2026-02-12T18:18:19.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj224 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.644 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.644 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.645 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj225 = obj250 ']' 2026-02-12T18:18:19.645 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj225 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.660 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.661 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.661 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj226 = obj250 ']' 2026-02-12T18:18:19.661 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj226 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.677 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.678 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.678 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj227 = obj250 ']' 2026-02-12T18:18:19.678 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj227 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.693 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.694 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.694 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj228 = obj250 ']' 2026-02-12T18:18:19.694 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj228 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.710 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.711 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.711 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj229 = obj250 ']' 2026-02-12T18:18:19.711 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj229 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.727 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.728 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj230 = obj250 ']' 2026-02-12T18:18:19.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj230 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.744 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.745 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.746 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj231 = obj250 ']' 2026-02-12T18:18:19.746 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj231 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.762 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj232 = obj250 ']' 2026-02-12T18:18:19.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj232 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.778 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.779 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.779 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj233 = obj250 ']' 2026-02-12T18:18:19.780 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj233 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.795 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.796 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.796 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj234 = obj250 ']' 2026-02-12T18:18:19.796 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj234 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.812 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.813 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.813 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj235 = obj250 ']' 2026-02-12T18:18:19.813 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj235 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.829 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.829 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj236 = obj250 ']' 2026-02-12T18:18:19.829 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj236 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.844 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.845 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.845 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj237 = obj250 ']' 2026-02-12T18:18:19.846 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj237 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.861 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.862 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.862 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj238 = obj250 ']' 2026-02-12T18:18:19.862 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj238 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.878 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.879 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.879 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj239 = obj250 ']' 2026-02-12T18:18:19.880 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj239 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.895 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.896 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.896 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj240 = obj250 ']' 2026-02-12T18:18:19.896 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj240 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.912 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.913 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.913 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj241 = obj250 ']' 2026-02-12T18:18:19.913 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj241 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.929 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.930 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj242 = obj250 ']' 2026-02-12T18:18:19.930 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj242 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.946 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.947 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.947 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj243 = obj250 ']' 2026-02-12T18:18:19.947 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj243 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.963 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.964 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.964 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj244 = obj250 ']' 2026-02-12T18:18:19.964 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj244 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.980 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.981 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.981 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj245 = obj250 ']' 2026-02-12T18:18:19.981 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj245 td/test-erasure-eio/CHECK 2026-02-12T18:18:19.997 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:19.998 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:19.998 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj246 = obj250 ']' 2026-02-12T18:18:19.998 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj246 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.014 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.015 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.015 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj247 = obj250 ']' 2026-02-12T18:18:20.015 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj247 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.031 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.032 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.032 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj248 = obj250 ']' 2026-02-12T18:18:20.032 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj248 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.048 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.048 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.049 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj249 = obj250 ']' 2026-02-12T18:18:20.049 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj249 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.065 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.066 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.066 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj250 = obj250 ']' 2026-02-12T18:18:20.066 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:615: TEST_ec_backfill_unfound: rados -p pool-jerasure get obj250 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.081 INFO:tasks.workunit.client.0.trial116.stderr:error getting pool-jerasure/obj250: (2) No such file or directory 2026-02-12T18:18:20.082 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.083 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj251 = obj250 ']' 2026-02-12T18:18:20.083 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj251 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.098 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.099 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.100 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj252 = obj250 ']' 2026-02-12T18:18:20.100 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj252 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.115 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.116 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.116 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj253 = obj250 ']' 2026-02-12T18:18:20.116 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj253 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.132 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.133 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.133 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj254 = obj250 ']' 2026-02-12T18:18:20.133 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj254 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.148 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.149 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.150 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj255 = obj250 ']' 2026-02-12T18:18:20.150 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj255 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.165 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.166 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.166 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj256 = obj250 ']' 2026-02-12T18:18:20.167 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj256 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.182 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.183 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.183 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj257 = obj250 ']' 2026-02-12T18:18:20.184 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj257 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.200 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.201 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.201 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj258 = obj250 ']' 2026-02-12T18:18:20.201 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj258 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.218 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.218 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj259 = obj250 ']' 2026-02-12T18:18:20.218 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj259 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.235 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.235 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj260 = obj250 ']' 2026-02-12T18:18:20.235 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj260 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.250 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.251 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.251 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj261 = obj250 ']' 2026-02-12T18:18:20.251 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj261 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.265 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.266 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.266 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj262 = obj250 ']' 2026-02-12T18:18:20.266 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj262 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.281 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.282 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.282 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj263 = obj250 ']' 2026-02-12T18:18:20.282 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj263 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.298 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj264 = obj250 ']' 2026-02-12T18:18:20.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj264 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.315 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.315 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj265 = obj250 ']' 2026-02-12T18:18:20.315 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj265 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.330 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.331 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.332 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj266 = obj250 ']' 2026-02-12T18:18:20.332 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj266 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.347 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.347 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj267 = obj250 ']' 2026-02-12T18:18:20.348 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj267 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.363 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.364 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.364 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj268 = obj250 ']' 2026-02-12T18:18:20.364 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj268 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.380 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj269 = obj250 ']' 2026-02-12T18:18:20.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj269 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.396 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.396 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.397 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj270 = obj250 ']' 2026-02-12T18:18:20.397 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj270 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.411 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.412 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.412 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj271 = obj250 ']' 2026-02-12T18:18:20.412 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj271 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.428 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.429 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj272 = obj250 ']' 2026-02-12T18:18:20.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj272 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.445 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.446 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.446 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj273 = obj250 ']' 2026-02-12T18:18:20.447 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj273 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.464 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.465 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.465 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj274 = obj250 ']' 2026-02-12T18:18:20.465 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj274 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.481 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.482 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.482 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj275 = obj250 ']' 2026-02-12T18:18:20.483 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj275 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.498 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj276 = obj250 ']' 2026-02-12T18:18:20.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj276 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.514 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.515 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.515 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj277 = obj250 ']' 2026-02-12T18:18:20.515 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj277 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.531 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.532 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.532 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj278 = obj250 ']' 2026-02-12T18:18:20.532 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj278 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.548 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.549 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.549 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj279 = obj250 ']' 2026-02-12T18:18:20.549 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj279 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.564 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.565 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.566 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj280 = obj250 ']' 2026-02-12T18:18:20.566 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj280 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.582 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.583 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.583 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj281 = obj250 ']' 2026-02-12T18:18:20.583 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj281 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.599 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.600 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.600 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj282 = obj250 ']' 2026-02-12T18:18:20.601 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj282 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.616 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.617 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.617 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj283 = obj250 ']' 2026-02-12T18:18:20.617 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj283 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.632 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.633 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.633 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj284 = obj250 ']' 2026-02-12T18:18:20.633 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj284 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.649 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.650 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.650 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj285 = obj250 ']' 2026-02-12T18:18:20.650 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj285 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.666 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.667 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.667 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj286 = obj250 ']' 2026-02-12T18:18:20.667 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj286 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.683 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.684 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.684 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj287 = obj250 ']' 2026-02-12T18:18:20.685 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj287 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.700 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.701 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.701 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj288 = obj250 ']' 2026-02-12T18:18:20.701 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj288 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.716 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.717 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.717 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj289 = obj250 ']' 2026-02-12T18:18:20.717 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj289 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.733 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.734 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.734 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj290 = obj250 ']' 2026-02-12T18:18:20.734 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj290 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.750 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.750 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj291 = obj250 ']' 2026-02-12T18:18:20.750 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj291 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.766 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.767 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.767 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj292 = obj250 ']' 2026-02-12T18:18:20.767 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj292 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.784 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj293 = obj250 ']' 2026-02-12T18:18:20.784 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj293 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.800 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj294 = obj250 ']' 2026-02-12T18:18:20.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj294 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.816 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.817 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.818 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj295 = obj250 ']' 2026-02-12T18:18:20.818 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj295 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.833 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.834 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.834 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj296 = obj250 ']' 2026-02-12T18:18:20.834 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj296 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.850 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.851 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.851 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj297 = obj250 ']' 2026-02-12T18:18:20.852 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj297 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.867 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.868 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.868 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj298 = obj250 ']' 2026-02-12T18:18:20.868 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj298 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.884 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.884 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj299 = obj250 ']' 2026-02-12T18:18:20.884 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj299 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.900 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.901 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:18:20.901 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj300 = obj250 ']' 2026-02-12T18:18:20.901 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj300 td/test-erasure-eio/CHECK 2026-02-12T18:18:20.916 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.917 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:622: TEST_ec_backfill_unfound: rm -f td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:18:20.918 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:624: TEST_ec_backfill_unfound: delete_erasure_coded_pool pool-jerasure 2026-02-12T18:18:20.918 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-12T18:18:20.918 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-12T18:18:21.193 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-jerasure' does not exist 2026-02-12T18:18:21.203 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-12T18:18:21.482 INFO:tasks.workunit.client.0.trial116.stderr:erasure-code-profile myprofile does not exist 2026-02-12T18:18:21.490 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-12T18:18:21.491 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-12T18:18:21.491 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:18:21.491 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-12T18:18:21.491 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:18:21.491 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:18:21.491 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:18:21.491 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:18:21.491 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:18:21.599 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:18:21.600 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:18:21.600 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:18:21.601 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:18:21.602 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:18:21.603 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:18:21.603 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:18:21.604 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:18:21.604 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:18:21.604 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:18:21.604 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:18:21.605 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:18:21.606 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:18:21.606 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-12T18:18:21.643 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:18:21.643 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:18:21.643 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:18:21.643 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76821 2026-02-12T18:18:21.645 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:18:21.645 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:18:21.645 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-12T18:18:21.645 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-12T18:18:21.646 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-12T18:18:21.646 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-12T18:18:21.646 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-12T18:18:21.646 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:18:21.646 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-12T18:18:21.646 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:18:21.646 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:18:21.646 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:18:21.646 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:18:21.646 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:18:21.647 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:18:21.647 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:18:21.648 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:18:21.648 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:18:21.649 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:18:21.649 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:18:21.649 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:18:21.650 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:18:21.650 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:18:21.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:18:21.651 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:18:21.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:18:21.652 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:18:21.652 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-12T18:18:21.653 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:18:21.654 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:18:21.654 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:18:21.654 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76821 2026-02-12T18:18:21.654 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:18:21.654 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:18:21.655 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-12T18:18:21.656 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T18:18:21.656 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:18:21.656 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:18:21.656 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76821 2026-02-12T18:18:21.657 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T18:18:21.658 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T18:18:21.658 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T18:18:21.658 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-12T18:18:21.658 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-12T18:18:21.658 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-12T18:18:21.658 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T18:18:21.658 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T18:18:21.659 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T18:18:21.659 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-12T18:18:21.659 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-12T18:18:21.676 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T18:18:21.676 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:18:21.676 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:18:21.676 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:18:21.676 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:18:21.676 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:18:21.676 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:18:21.677 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T18:18:21.700 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T18:18:21.701 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T18:18:21.701 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:18:21.701 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:18:21.701 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T18:18:21.701 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T18:18:21.701 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:18:21.702 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:18:21.702 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:18:21.702 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:18:21.703 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:18:21.703 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:18:21.703 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:18:21.703 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:18:21.703 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76821/ceph-mon.a.asok config get fsid 2026-02-12T18:18:21.745 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T18:18:21.745 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:18:21.745 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:18:21.745 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T18:18:21.746 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T18:18:21.747 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:18:21.747 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:18:21.747 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:18:21.747 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:18:21.747 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:18:21.747 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:18:21.747 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:18:21.747 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:18:21.747 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76821/ceph-mon.a.asok config get mon_host 2026-02-12T18:18:21.789 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-12T18:18:21.789 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-12T18:18:21.790 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T18:18:21.790 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T18:18:21.790 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T18:18:21.790 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-12T18:18:21.790 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T18:18:21.892 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T18:18:21.892 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:18:21.892 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:18:21.892 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:18:21.892 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:18:21.892 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:18:21.892 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:18:21.893 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:18:21.893 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:18:21.908 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-12T18:18:21.908 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T18:18:22.003 INFO:tasks.workunit.client.0.trial116.stderr:pool 'rbd' already exists 2026-02-12T18:18:22.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:18:23.013 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-12T18:18:23.013 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:18:23.013 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:18:23.013 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:18:23.013 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:18:23.013 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:18:23.013 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:18:23.013 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-12T18:18:23.013 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-mon.a.asok log flush 2026-02-12T18:18:23.053 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:18:23.058 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-12T18:18:23.060 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:18:21.689+0000 7f644ce57c00 0 load: jerasure load: lrc 2026-02-12T18:18:23.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_object_attr_read_error td/test-erasure-eio 2026-02-12T18:18:23.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:379: TEST_ec_object_attr_read_error: local dir=td/test-erasure-eio 2026-02-12T18:18:23.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:380: TEST_ec_object_attr_read_error: local objname=myobject 2026-02-12T18:18:23.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:382: TEST_ec_object_attr_read_error: setup_osds 7 2026-02-12T18:18:23.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-12T18:18:23.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-12T18:18:23.061 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-12T18:18:23.065 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-12T18:18:23.065 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:18:23.065 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-12T18:18:23.065 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:18:23.065 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:18:23.066 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T18:18:23.066 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:18:23.066 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-12T18:18:23.066 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:18:23.066 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:18:23.066 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:18:23.066 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:18:23.066 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-12T18:18:23.066 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-12T18:18:23.066 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:18:23.066 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:18:23.067 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:18:23.067 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:18:23.067 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:18:23.067 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:18:23.067 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:18:23.067 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:18:23.067 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:18:23.067 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:18:23.067 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:18:23.067 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:18:23.067 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:18:23.067 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:18:23.067 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:18:23.068 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:18:23.068 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:18:23.068 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:18:23.068 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:18:23.068 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:18:23.068 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:18:23.068 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:18:23.068 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-12T18:18:23.068 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:18:23.068 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=dfe9463a-25e5-47c3-98c3-da621736f38f 2026-02-12T18:18:23.068 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 dfe9463a-25e5-47c3-98c3-da621736f38f' 2026-02-12T18:18:23.069 INFO:tasks.workunit.client.0.trial116.stdout:add osd0 dfe9463a-25e5-47c3-98c3-da621736f38f 2026-02-12T18:18:23.069 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:18:23.075 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBvGY5p/t5IBBAAtblc8RDoVrqt5yp5HI7cUw== 2026-02-12T18:18:23.075 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBvGY5p/t5IBBAAtblc8RDoVrqt5yp5HI7cUw=="}' 2026-02-12T18:18:23.075 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new dfe9463a-25e5-47c3-98c3-da621736f38f -i td/test-erasure-eio/0/new.json 2026-02-12T18:18:23.179 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:18:23.185 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-12T18:18:23.187 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBvGY5p/t5IBBAAtblc8RDoVrqt5yp5HI7cUw== --osd-uuid dfe9463a-25e5-47c3-98c3-da621736f38f 2026-02-12T18:18:23.203 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:23.199+0000 7f03333d4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:23.204 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:23.200+0000 7f03333d4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:23.205 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:23.201+0000 7f03333d4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:23.206 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:23.202+0000 7f03333d4780 -1 bdev(0x4140c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-12T18:18:23.206 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:23.202+0000 7f03333d4780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-12T18:18:23.655 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-12T18:18:23.655 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:18:23.655 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T18:18:23.656 INFO:tasks.workunit.client.0.trial116.stdout:adding osd0 key to auth repository 2026-02-12T18:18:23.656 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:18:23.761 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T18:18:23.762 INFO:tasks.workunit.client.0.trial116.stdout:start osd.0 2026-02-12T18:18:23.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:18:23.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:18:23.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:18:23.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:18:23.777 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:23.773+0000 7f425502f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:23.778 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:23.775+0000 7f425502f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:23.779 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:23.776+0000 7f425502f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:23.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T18:18:23.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:18:23.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T18:18:23.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:18:23.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:18:23.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:18:23.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:18:23.859 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:18:23.860 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:18:23.860 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:18:23.954 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:18:23.984 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:23.980+0000 7f425502f780 -1 Falling back to public interface 2026-02-12T18:18:24.243 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:24.240+0000 7f425502f780 -1 osd.0 0 log_to_monitors true 2026-02-12T18:18:24.955 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:18:24.955 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:18:24.955 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:18:24.955 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:18:24.955 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:18:24.956 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:18:25.195 INFO:tasks.workunit.client.0.trial116.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3778057127,v1:127.0.0.1:6803/3778057127] [v2:127.0.0.1:6804/3778057127,v1:127.0.0.1:6805/3778057127] exists,up dfe9463a-25e5-47c3-98c3-da621736f38f 2026-02-12T18:18:25.195 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:18:25.195 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:18:25.195 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:18:25.195 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:18:25.195 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-12T18:18:25.195 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:18:25.195 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:18:25.195 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-12T18:18:25.195 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:18:25.195 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-12T18:18:25.195 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:18:25.195 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:18:25.195 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:18:25.195 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:18:25.195 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-12T18:18:25.196 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-12T18:18:25.196 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:18:25.196 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:18:25.196 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:18:25.198 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:18:25.198 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:18:25.198 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:18:25.198 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:18:25.198 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:18:25.198 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:18:25.198 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:18:25.198 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:18:25.198 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:18:25.198 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:18:25.199 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:18:25.199 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:18:25.199 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:18:25.199 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:18:25.199 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:18:25.199 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:18:25.199 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:18:25.199 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:18:25.199 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:18:25.199 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-12T18:18:25.200 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:18:25.200 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=de9ddcea-09eb-49d7-8f76-3900d3f280ec 2026-02-12T18:18:25.200 INFO:tasks.workunit.client.0.trial116.stdout:add osd1 de9ddcea-09eb-49d7-8f76-3900d3f280ec 2026-02-12T18:18:25.201 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 de9ddcea-09eb-49d7-8f76-3900d3f280ec' 2026-02-12T18:18:25.201 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:18:25.209 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBxGY5pUg1EDBAAqb2Wc/kVG9T67clZXPEUQQ== 2026-02-12T18:18:25.209 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBxGY5pUg1EDBAAqb2Wc/kVG9T67clZXPEUQQ=="}' 2026-02-12T18:18:25.209 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new de9ddcea-09eb-49d7-8f76-3900d3f280ec -i td/test-erasure-eio/1/new.json 2026-02-12T18:18:25.439 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:18:25.447 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-12T18:18:25.448 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBxGY5pUg1EDBAAqb2Wc/kVG9T67clZXPEUQQ== --osd-uuid de9ddcea-09eb-49d7-8f76-3900d3f280ec 2026-02-12T18:18:25.463 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:25.460+0000 7f0aa8e10780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:25.465 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:25.461+0000 7f0aa8e10780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:25.466 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:25.462+0000 7f0aa8e10780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:25.466 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:25.462+0000 7f0aa8e10780 -1 bdev(0x409ec00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-12T18:18:25.466 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:25.462+0000 7f0aa8e10780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-12T18:18:25.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-12T18:18:25.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:18:25.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-12T18:18:25.783 INFO:tasks.workunit.client.0.trial116.stdout:adding osd1 key to auth repository 2026-02-12T18:18:25.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:18:26.108 INFO:tasks.workunit.client.0.trial116.stdout:start osd.1 2026-02-12T18:18:26.109 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-12T18:18:26.109 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:18:26.109 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:18:26.109 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:18:26.109 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:18:26.123 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:26.119+0000 7fcd43e2c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:26.124 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:26.120+0000 7fcd43e2c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:26.125 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:26.121+0000 7fcd43e2c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:26.247 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:26.244+0000 7fcd43e2c780 -1 Falling back to public interface 2026-02-12T18:18:26.333 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-12T18:18:26.333 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:18:26.333 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T18:18:26.334 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:18:26.334 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:18:26.334 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:18:26.334 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:18:26.334 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:18:26.335 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:18:26.335 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:18:26.565 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:26.561+0000 7fcd43e2c780 -1 osd.1 0 log_to_monitors true 2026-02-12T18:18:26.571 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:18:27.572 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:18:27.572 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:18:27.573 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:18:27.573 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:18:27.574 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:18:27.574 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:18:27.797 INFO:tasks.workunit.client.0.trial116.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2309689992,v1:127.0.0.1:6811/2309689992] [v2:127.0.0.1:6812/2309689992,v1:127.0.0.1:6813/2309689992] exists,up de9ddcea-09eb-49d7-8f76-3900d3f280ec 2026-02-12T18:18:27.800 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:18:27.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:18:27.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:18:27.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:18:27.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-12T18:18:27.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:18:27.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:18:27.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-12T18:18:27.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:18:27.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-12T18:18:27.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:18:27.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:18:27.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:18:27.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:18:27.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-12T18:18:27.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-12T18:18:27.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:18:27.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:18:27.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:18:27.802 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:18:27.802 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:18:27.802 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:18:27.802 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:18:27.803 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:18:27.803 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:18:27.803 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:18:27.803 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:18:27.803 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:18:27.803 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:18:27.803 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:18:27.803 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:18:27.803 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:18:27.803 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:18:27.803 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:18:27.803 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:18:27.803 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:18:27.804 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:18:27.804 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:18:27.804 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-12T18:18:27.804 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:18:27.804 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=93e2f73c-2935-44c7-8360-9d31cc79b2ee 2026-02-12T18:18:27.804 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 93e2f73c-2935-44c7-8360-9d31cc79b2ee' 2026-02-12T18:18:27.804 INFO:tasks.workunit.client.0.trial116.stdout:add osd2 93e2f73c-2935-44c7-8360-9d31cc79b2ee 2026-02-12T18:18:27.805 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:18:27.810 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBzGY5pWuwcMBAASbquV8YzQ/lkKwARf8G/0Q== 2026-02-12T18:18:27.810 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBzGY5pWuwcMBAASbquV8YzQ/lkKwARf8G/0Q=="}' 2026-02-12T18:18:27.810 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 93e2f73c-2935-44c7-8360-9d31cc79b2ee -i td/test-erasure-eio/2/new.json 2026-02-12T18:18:28.044 INFO:tasks.workunit.client.0.trial116.stdout:2 2026-02-12T18:18:28.051 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-12T18:18:28.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBzGY5pWuwcMBAASbquV8YzQ/lkKwARf8G/0Q== --osd-uuid 93e2f73c-2935-44c7-8360-9d31cc79b2ee 2026-02-12T18:18:28.068 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:28.066+0000 7fcfeb00e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:28.070 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:28.067+0000 7fcfeb00e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:28.071 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:28.068+0000 7fcfeb00e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:28.071 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:28.068+0000 7fcfeb00e780 -1 bdev(0x32eec00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-12T18:18:28.071 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:28.068+0000 7fcfeb00e780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-12T18:18:28.392 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-12T18:18:28.392 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:18:28.393 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-12T18:18:28.393 INFO:tasks.workunit.client.0.trial116.stdout:adding osd2 key to auth repository 2026-02-12T18:18:28.393 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:18:28.736 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-12T18:18:28.736 INFO:tasks.workunit.client.0.trial116.stdout:start osd.2 2026-02-12T18:18:28.737 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:18:28.737 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:18:28.737 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:18:28.737 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:18:28.751 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:28.748+0000 7f17959df780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:28.752 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:28.750+0000 7f17959df780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:28.753 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:28.751+0000 7f17959df780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:28.875 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:28.873+0000 7f17959df780 -1 Falling back to public interface 2026-02-12T18:18:28.958 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-12T18:18:28.959 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:18:28.959 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T18:18:28.959 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:18:28.959 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:18:28.959 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:18:28.959 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:18:28.959 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:18:28.960 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:18:28.960 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:18:29.166 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:29.164+0000 7f17959df780 -1 osd.2 0 log_to_monitors true 2026-02-12T18:18:29.189 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:18:30.190 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:18:30.191 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:18:30.191 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:18:30.192 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:18:30.192 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:18:30.192 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:18:30.424 INFO:tasks.workunit.client.0.trial116.stdout:osd.2 up in weight 1 up_from 18 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2487521568,v1:127.0.0.1:6819/2487521568] [v2:127.0.0.1:6820/2487521568,v1:127.0.0.1:6821/2487521568] exists,up 93e2f73c-2935-44c7-8360-9d31cc79b2ee 2026-02-12T18:18:30.428 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:18:30.429 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:18:30.429 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:18:30.429 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:18:30.429 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-12T18:18:30.429 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:18:30.429 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:18:30.429 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-12T18:18:30.429 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:18:30.429 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-12T18:18:30.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:18:30.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:18:30.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:18:30.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:18:30.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-12T18:18:30.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-12T18:18:30.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:18:30.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:18:30.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:18:30.430 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:18:30.430 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:18:30.430 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:18:30.431 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:18:30.431 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:18:30.431 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:18:30.431 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:18:30.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:18:30.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:18:30.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:18:30.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:18:30.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:18:30.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:18:30.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:18:30.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:18:30.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:18:30.432 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:18:30.432 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:18:30.432 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:18:30.432 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-12T18:18:30.432 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:18:30.432 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d5e6f499-26c5-4213-ae9b-e703ab0c3efb 2026-02-12T18:18:30.432 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 d5e6f499-26c5-4213-ae9b-e703ab0c3efb' 2026-02-12T18:18:30.432 INFO:tasks.workunit.client.0.trial116.stdout:add osd3 d5e6f499-26c5-4213-ae9b-e703ab0c3efb 2026-02-12T18:18:30.433 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:18:30.439 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB2GY5pioH4GRAAf2J1i5ntxD4RsL1DkUvrww== 2026-02-12T18:18:30.439 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB2GY5pioH4GRAAf2J1i5ntxD4RsL1DkUvrww=="}' 2026-02-12T18:18:30.439 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d5e6f499-26c5-4213-ae9b-e703ab0c3efb -i td/test-erasure-eio/3/new.json 2026-02-12T18:18:30.674 INFO:tasks.workunit.client.0.trial116.stdout:3 2026-02-12T18:18:30.681 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-12T18:18:30.682 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB2GY5pioH4GRAAf2J1i5ntxD4RsL1DkUvrww== --osd-uuid d5e6f499-26c5-4213-ae9b-e703ab0c3efb 2026-02-12T18:18:30.699 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:30.696+0000 7fafa6c13780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:30.700 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:30.698+0000 7fafa6c13780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:30.701 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:30.698+0000 7fafa6c13780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:30.702 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:30.699+0000 7fafa6c13780 -1 bdev(0x2446c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-12T18:18:30.702 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:30.699+0000 7fafa6c13780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-12T18:18:31.036 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-12T18:18:31.036 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:18:31.037 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-12T18:18:31.037 INFO:tasks.workunit.client.0.trial116.stdout:adding osd3 key to auth repository 2026-02-12T18:18:31.038 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:18:31.364 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-12T18:18:31.365 INFO:tasks.workunit.client.0.trial116.stdout:start osd.3 2026-02-12T18:18:31.365 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:18:31.365 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:18:31.366 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:18:31.366 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:18:31.379 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:31.377+0000 7f17c402d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:31.381 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:31.378+0000 7f17c402d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:31.382 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:31.379+0000 7f17c402d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:31.539 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:31.537+0000 7f17c402d780 -1 Falling back to public interface 2026-02-12T18:18:31.590 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-12T18:18:31.590 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:18:31.590 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-12T18:18:31.590 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:18:31.590 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:18:31.591 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:18:31.591 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:18:31.591 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:18:31.591 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:18:31.591 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:18:31.815 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:18:31.874 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:31.871+0000 7f17c402d780 -1 osd.3 0 log_to_monitors true 2026-02-12T18:18:32.816 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:18:32.817 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:18:32.817 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:18:32.818 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:18:32.818 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:18:32.818 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:18:33.045 INFO:tasks.workunit.client.0.trial116.stdout:osd.3 up in weight 1 up_from 27 up_thru 28 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1934036709,v1:127.0.0.1:6827/1934036709] [v2:127.0.0.1:6828/1934036709,v1:127.0.0.1:6829/1934036709] exists,up d5e6f499-26c5-4213-ae9b-e703ab0c3efb 2026-02-12T18:18:33.049 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:18:33.050 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:18:33.050 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:18:33.050 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:18:33.050 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-12T18:18:33.050 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:18:33.050 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:18:33.050 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-12T18:18:33.050 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:18:33.050 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-12T18:18:33.051 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:18:33.051 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:18:33.051 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:18:33.051 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:18:33.051 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-12T18:18:33.051 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-12T18:18:33.051 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:18:33.051 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:18:33.051 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:18:33.051 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:18:33.051 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:18:33.052 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:18:33.052 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:18:33.052 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:18:33.052 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:18:33.052 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:18:33.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:18:33.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:18:33.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:18:33.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:18:33.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:18:33.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:18:33.053 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:18:33.053 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:18:33.053 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:18:33.053 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:18:33.053 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:18:33.053 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:18:33.053 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-12T18:18:33.053 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:18:33.053 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a07ca641-45ba-4e9c-87e0-6daf6c38e440 2026-02-12T18:18:33.053 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 a07ca641-45ba-4e9c-87e0-6daf6c38e440' 2026-02-12T18:18:33.054 INFO:tasks.workunit.client.0.trial116.stdout:add osd4 a07ca641-45ba-4e9c-87e0-6daf6c38e440 2026-02-12T18:18:33.054 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:18:33.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB5GY5pfU9cAxAAOLd22UrWrqrG2cTcNcmsAQ== 2026-02-12T18:18:33.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB5GY5pfU9cAxAAOLd22UrWrqrG2cTcNcmsAQ=="}' 2026-02-12T18:18:33.060 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a07ca641-45ba-4e9c-87e0-6daf6c38e440 -i td/test-erasure-eio/4/new.json 2026-02-12T18:18:33.291 INFO:tasks.workunit.client.0.trial116.stdout:4 2026-02-12T18:18:33.298 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-12T18:18:33.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB5GY5pfU9cAxAAOLd22UrWrqrG2cTcNcmsAQ== --osd-uuid a07ca641-45ba-4e9c-87e0-6daf6c38e440 2026-02-12T18:18:33.314 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:33.311+0000 7f09f9655780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:33.315 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:33.313+0000 7f09f9655780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:33.316 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:33.313+0000 7f09f9655780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:33.317 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:33.314+0000 7f09f9655780 -1 bdev(0x26b8c00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-12T18:18:33.317 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:33.314+0000 7f09f9655780 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-12T18:18:33.641 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-12T18:18:33.641 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:18:33.641 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-12T18:18:33.642 INFO:tasks.workunit.client.0.trial116.stdout:adding osd4 key to auth repository 2026-02-12T18:18:33.642 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:18:33.957 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-12T18:18:33.958 INFO:tasks.workunit.client.0.trial116.stdout:start osd.4 2026-02-12T18:18:33.958 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:18:33.959 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:18:33.959 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:18:33.959 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:18:33.971 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:33.968+0000 7fb84b63a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:33.972 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:33.970+0000 7fb84b63a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:33.973 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:33.970+0000 7fb84b63a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:34.169 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:34.166+0000 7fb84b63a780 -1 Falling back to public interface 2026-02-12T18:18:34.183 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-12T18:18:34.183 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:18:34.184 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-12T18:18:34.184 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:18:34.184 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:18:34.184 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:18:34.184 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:18:34.184 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:18:34.184 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:18:34.184 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-12T18:18:34.406 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:18:34.512 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:34.509+0000 7fb84b63a780 -1 osd.4 0 log_to_monitors true 2026-02-12T18:18:35.407 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:18:35.407 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:18:35.407 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:18:35.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:18:35.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:18:35.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-12T18:18:35.633 INFO:tasks.workunit.client.0.trial116.stdout:osd.4 up in weight 1 up_from 36 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/2051784967,v1:127.0.0.1:6835/2051784967] [v2:127.0.0.1:6836/2051784967,v1:127.0.0.1:6837/2051784967] exists,up a07ca641-45ba-4e9c-87e0-6daf6c38e440 2026-02-12T18:18:35.637 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:18:35.637 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:18:35.637 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:18:35.637 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:18:35.637 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-12T18:18:35.637 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:18:35.637 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:18:35.637 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-12T18:18:35.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:18:35.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-12T18:18:35.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:18:35.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:18:35.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:18:35.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:18:35.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-12T18:18:35.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-12T18:18:35.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:18:35.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:18:35.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:18:35.638 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:18:35.639 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:18:35.639 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:18:35.639 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:18:35.639 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:18:35.639 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:18:35.639 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:18:35.639 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:18:35.639 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:18:35.639 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:18:35.639 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:18:35.639 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:18:35.639 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:18:35.639 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:18:35.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:18:35.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:18:35.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:18:35.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:18:35.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:18:35.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-12T18:18:35.640 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:18:35.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f5ce4ca0-fe3d-4c56-acba-10a6f538c77f 2026-02-12T18:18:35.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 f5ce4ca0-fe3d-4c56-acba-10a6f538c77f' 2026-02-12T18:18:35.641 INFO:tasks.workunit.client.0.trial116.stdout:add osd5 f5ce4ca0-fe3d-4c56-acba-10a6f538c77f 2026-02-12T18:18:35.641 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:18:35.647 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB7GY5p9mhiJhAA/Qs9jMWXfHyz1V1AlhsY4Q== 2026-02-12T18:18:35.647 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB7GY5p9mhiJhAA/Qs9jMWXfHyz1V1AlhsY4Q=="}' 2026-02-12T18:18:35.647 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f5ce4ca0-fe3d-4c56-acba-10a6f538c77f -i td/test-erasure-eio/5/new.json 2026-02-12T18:18:35.882 INFO:tasks.workunit.client.0.trial116.stdout:5 2026-02-12T18:18:35.891 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-12T18:18:35.892 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB7GY5p9mhiJhAA/Qs9jMWXfHyz1V1AlhsY4Q== --osd-uuid f5ce4ca0-fe3d-4c56-acba-10a6f538c77f 2026-02-12T18:18:35.907 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:35.905+0000 7f0cc3e6f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:35.909 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:35.906+0000 7f0cc3e6f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:35.909 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:35.907+0000 7f0cc3e6f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:35.910 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:35.907+0000 7f0cc3e6f780 -1 bdev(0x28dcc00 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-12T18:18:35.910 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:35.907+0000 7f0cc3e6f780 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-12T18:18:36.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-12T18:18:36.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:18:36.245 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-12T18:18:36.245 INFO:tasks.workunit.client.0.trial116.stdout:adding osd5 key to auth repository 2026-02-12T18:18:36.245 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:18:36.585 INFO:tasks.workunit.client.0.trial116.stdout:start osd.5 2026-02-12T18:18:36.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-12T18:18:36.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:18:36.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:18:36.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:18:36.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:18:36.599 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:36.597+0000 7fced6c1e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:36.601 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:36.598+0000 7fced6c1e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:36.601 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:36.599+0000 7fced6c1e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:36.762 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:36.759+0000 7fced6c1e780 -1 Falling back to public interface 2026-02-12T18:18:36.814 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-12T18:18:36.814 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:18:36.814 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-12T18:18:36.814 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:18:36.814 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:18:36.814 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:18:36.815 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:18:36.815 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:18:36.815 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:18:36.815 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-12T18:18:37.043 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:18:37.081 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:37.078+0000 7fced6c1e780 -1 osd.5 0 log_to_monitors true 2026-02-12T18:18:38.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:18:38.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:18:38.045 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:18:38.045 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:18:38.046 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:18:38.046 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-12T18:18:38.273 INFO:tasks.workunit.client.0.trial116.stdout:osd.5 up in weight 1 up_from 43 up_thru 43 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/491003527,v1:127.0.0.1:6843/491003527] [v2:127.0.0.1:6844/491003527,v1:127.0.0.1:6845/491003527] exists,up f5ce4ca0-fe3d-4c56-acba-10a6f538c77f 2026-02-12T18:18:38.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:18:38.278 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:18:38.278 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:18:38.278 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:18:38.278 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-12T18:18:38.278 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:18:38.278 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:18:38.278 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-12T18:18:38.278 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:18:38.278 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-12T18:18:38.279 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:18:38.279 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:18:38.279 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:18:38.279 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:18:38.279 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-12T18:18:38.279 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-12T18:18:38.279 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:18:38.279 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:18:38.279 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:18:38.279 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:18:38.279 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:18:38.279 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:18:38.280 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:18:38.280 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:18:38.280 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:18:38.280 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:18:38.280 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:18:38.280 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:18:38.280 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:18:38.280 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:18:38.280 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:18:38.280 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:18:38.280 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:18:38.281 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:18:38.281 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:18:38.281 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:18:38.281 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:18:38.281 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:18:38.281 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-12T18:18:38.281 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:18:38.281 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=74f77dc6-7ddc-4344-81e1-249b8ff3f3d2 2026-02-12T18:18:38.281 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 74f77dc6-7ddc-4344-81e1-249b8ff3f3d2' 2026-02-12T18:18:38.282 INFO:tasks.workunit.client.0.trial116.stdout:add osd6 74f77dc6-7ddc-4344-81e1-249b8ff3f3d2 2026-02-12T18:18:38.282 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:18:38.287 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB+GY5peNrvEBAA5gWvdf23fBiqlz5J0cyHgQ== 2026-02-12T18:18:38.287 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB+GY5peNrvEBAA5gWvdf23fBiqlz5J0cyHgQ=="}' 2026-02-12T18:18:38.287 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 74f77dc6-7ddc-4344-81e1-249b8ff3f3d2 -i td/test-erasure-eio/6/new.json 2026-02-12T18:18:38.520 INFO:tasks.workunit.client.0.trial116.stdout:6 2026-02-12T18:18:38.528 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-12T18:18:38.529 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB+GY5peNrvEBAA5gWvdf23fBiqlz5J0cyHgQ== --osd-uuid 74f77dc6-7ddc-4344-81e1-249b8ff3f3d2 2026-02-12T18:18:38.545 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:38.541+0000 7fb811a36780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:38.546 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:38.542+0000 7fb811a36780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:38.547 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:38.543+0000 7fb811a36780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:38.547 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:38.543+0000 7fb811a36780 -1 bdev(0x2696c00 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-12T18:18:38.548 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:38.544+0000 7fb811a36780 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-12T18:18:38.869 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-12T18:18:38.869 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:18:38.870 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-12T18:18:38.870 INFO:tasks.workunit.client.0.trial116.stdout:adding osd6 key to auth repository 2026-02-12T18:18:38.870 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:18:39.195 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-12T18:18:39.195 INFO:tasks.workunit.client.0.trial116.stdout:start osd.6 2026-02-12T18:18:39.196 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:18:39.196 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:18:39.197 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:18:39.197 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:18:39.209 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:39.205+0000 7fb29cfd8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:39.210 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:39.207+0000 7fb29cfd8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:39.211 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:39.207+0000 7fb29cfd8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:39.344 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:39.340+0000 7fb29cfd8780 -1 Falling back to public interface 2026-02-12T18:18:39.418 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-12T18:18:39.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:18:39.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-12T18:18:39.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:18:39.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:18:39.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:18:39.419 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:18:39.420 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:18:39.420 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:18:39.420 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-12T18:18:39.642 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:39.638+0000 7fb29cfd8780 -1 osd.6 0 log_to_monitors true 2026-02-12T18:18:39.645 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:18:40.646 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:18:40.647 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:18:40.647 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:18:40.648 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:18:40.648 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:18:40.648 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-12T18:18:40.868 INFO:tasks.workunit.client.0.trial116.stdout:osd.6 up in weight 1 up_from 51 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/64377654,v1:127.0.0.1:6851/64377654] [v2:127.0.0.1:6852/64377654,v1:127.0.0.1:6853/64377654] exists,up 74f77dc6-7ddc-4344-81e1-249b8ff3f3d2 2026-02-12T18:18:40.870 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:18:40.870 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:18:40.871 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:18:40.871 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-12T18:18:40.871 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T18:18:40.871 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T18:18:40.871 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:18:40.871 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:18:40.871 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:18:40.871 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.0.asok 2026-02-12T18:18:40.871 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-12T18:18:40.871 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-osd.0.asok log flush 2026-02-12T18:18:40.915 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:18:40.919 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-12T18:18:40.922 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:18:23.983+0000 7f425502f780 0 load: jerasure load: lrc 2026-02-12T18:18:40.923 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:384: TEST_ec_object_attr_read_error: local poolname=pool-jerasure 2026-02-12T18:18:40.923 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:385: TEST_ec_object_attr_read_error: create_erasure_coded_pool pool-jerasure 3 2 2026-02-12T18:18:40.923 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-12T18:18:40.923 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-12T18:18:40.923 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-12T18:18:40.923 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-12T18:18:40.924 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-12T18:18:40.924 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-12T18:18:40.924 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-12T18:18:41.232 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-12T18:18:41.232 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-12T18:18:41.609 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-jerasure' already exists 2026-02-12T18:18:41.617 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:18:42.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-12T18:18:42.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:18:42.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:18:42.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:18:42.620 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:18:42.620 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:18:42.621 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:18:42.621 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:18:42.621 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:18:42.621 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:18:42.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:18:42.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:18:42.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:18:42.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:18:42.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:18:42.706 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:18:42.932 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:18:42.932 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:18:42.932 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:18:42.932 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:18:42.932 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:18:42.932 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:18:42.932 INFO:tasks.workunit.client.0.trial116.stderr:6' 2026-02-12T18:18:42.932 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:18:42.933 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:18:42.933 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:18:42.997 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803782 2026-02-12T18:18:42.997 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803782 2026-02-12T18:18:42.998 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782' 2026-02-12T18:18:42.998 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:18:42.998 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:18:43.060 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574854 2026-02-12T18:18:43.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574854 2026-02-12T18:18:43.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574854' 2026-02-12T18:18:43.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:18:43.061 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:18:43.125 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411334 2026-02-12T18:18:43.125 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411334 2026-02-12T18:18:43.125 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-77309411334' 2026-02-12T18:18:43.125 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:18:43.126 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:18:43.189 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116998 2026-02-12T18:18:43.189 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116998 2026-02-12T18:18:43.189 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-77309411334 3-115964116998' 2026-02-12T18:18:43.189 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:18:43.189 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-12T18:18:43.255 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=154618822660 2026-02-12T18:18:43.255 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 154618822660 2026-02-12T18:18:43.255 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-77309411334 3-115964116998 4-154618822660' 2026-02-12T18:18:43.255 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:18:43.255 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-12T18:18:43.319 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593732 2026-02-12T18:18:43.319 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593732 2026-02-12T18:18:43.319 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-77309411334 3-115964116998 4-154618822660 5-184683593732' 2026-02-12T18:18:43.319 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:18:43.319 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-12T18:18:43.384 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=219043332100 2026-02-12T18:18:43.384 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 219043332100 2026-02-12T18:18:43.384 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-77309411334 3-115964116998 4-154618822660 5-184683593732 6-219043332100' 2026-02-12T18:18:43.384 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:18:43.385 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803782 2026-02-12T18:18:43.385 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:18:43.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:18:43.386 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803782 2026-02-12T18:18:43.386 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:18:43.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803782 2026-02-12T18:18:43.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803782' 2026-02-12T18:18:43.388 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.0 seq 25769803782 2026-02-12T18:18:43.388 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:18:43.615 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803780 -lt 25769803782 2026-02-12T18:18:43.615 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:18:44.616 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:18:44.617 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:18:44.846 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803782 2026-02-12T18:18:44.846 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:18:44.846 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574854 2026-02-12T18:18:44.847 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:18:44.847 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:18:44.848 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574854 2026-02-12T18:18:44.848 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:18:44.849 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574854 2026-02-12T18:18:44.849 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574854' 2026-02-12T18:18:44.849 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.1 seq 55834574854 2026-02-12T18:18:44.850 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:18:45.078 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574854 -lt 55834574854 2026-02-12T18:18:45.078 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:18:45.079 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411334 2026-02-12T18:18:45.079 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:18:45.079 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:18:45.080 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411334 2026-02-12T18:18:45.080 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:18:45.081 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411334 2026-02-12T18:18:45.082 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411334' 2026-02-12T18:18:45.082 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.2 seq 77309411334 2026-02-12T18:18:45.082 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:18:45.309 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411334 -lt 77309411334 2026-02-12T18:18:45.309 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:18:45.310 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-115964116998 2026-02-12T18:18:45.310 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:18:45.311 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:18:45.311 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964116998 2026-02-12T18:18:45.312 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:18:45.313 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964116998 2026-02-12T18:18:45.313 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 115964116998' 2026-02-12T18:18:45.313 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.3 seq 115964116998 2026-02-12T18:18:45.313 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:18:45.538 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964116998 -lt 115964116998 2026-02-12T18:18:45.538 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:18:45.538 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-154618822660 2026-02-12T18:18:45.539 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:18:45.539 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-12T18:18:45.540 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-154618822660 2026-02-12T18:18:45.540 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:18:45.541 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=154618822660 2026-02-12T18:18:45.541 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.4 seq 154618822660 2026-02-12T18:18:45.542 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 154618822660' 2026-02-12T18:18:45.542 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-12T18:18:45.773 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 154618822660 -lt 154618822660 2026-02-12T18:18:45.773 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:18:45.774 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-184683593732 2026-02-12T18:18:45.774 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:18:45.775 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-12T18:18:45.775 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-184683593732 2026-02-12T18:18:45.775 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:18:45.776 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593732 2026-02-12T18:18:45.776 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 184683593732' 2026-02-12T18:18:45.777 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.5 seq 184683593732 2026-02-12T18:18:45.777 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-12T18:18:46.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593732 -lt 184683593732 2026-02-12T18:18:46.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:18:46.005 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-219043332100 2026-02-12T18:18:46.005 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:18:46.005 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-12T18:18:46.006 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-219043332100 2026-02-12T18:18:46.006 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:18:46.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=219043332100 2026-02-12T18:18:46.007 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.6 seq 219043332100 2026-02-12T18:18:46.008 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 219043332100' 2026-02-12T18:18:46.008 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-12T18:18:46.230 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 219043332101 -lt 219043332100 2026-02-12T18:18:46.231 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:18:46.231 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:18:46.232 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:18:46.548 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:18:46.548 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:18:46.548 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:18:46.548 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:18:46.549 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:18:46.549 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:18:46.549 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:18:46.549 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:18:46.769 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:18:46.770 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:18:46.770 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:18:46.770 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:18:47.085 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:18:47.085 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:18:47.085 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:18:47.085 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:387: TEST_ec_object_attr_read_error: get_primary pool-jerasure myobject 2026-02-12T18:18:47.085 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=pool-jerasure 2026-02-12T18:18:47.085 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=myobject 2026-02-12T18:18:47.085 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map pool-jerasure myobject 2026-02-12T18:18:47.085 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-12T18:18:47.311 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:387: TEST_ec_object_attr_read_error: local primary_osd=3 2026-02-12T18:18:47.311 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:389: TEST_ec_object_attr_read_error: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-12T18:18:47.312 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:18:47.312 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:18:47.312 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:18:47.312 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:18:47.312 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:18:47.418 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:18:47.418 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:392: TEST_ec_object_attr_read_error: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-12T18:18:47.418 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-12T18:18:47.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-12T18:18:47.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-12T18:18:47.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:18:47.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-12T18:18:47.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:18:47.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-12T18:18:47.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:18:47.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-12T18:18:47.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:18:47.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-12T18:18:47.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-12T18:18:48.251 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:395: TEST_ec_object_attr_read_error: inject_eio ec mdata pool-jerasure myobject td/test-erasure-eio 1 2026-02-12T18:18:48.251 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-12T18:18:48.251 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-12T18:18:48.251 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=mdata 2026-02-12T18:18:48.251 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-12T18:18:48.251 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-12T18:18:48.252 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-12T18:18:48.252 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=myobject 2026-02-12T18:18:48.252 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-12T18:18:48.252 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-12T18:18:48.252 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-12T18:18:48.252 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=1 2026-02-12T18:18:48.252 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-12T18:18:48.252 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure myobject 2026-02-12T18:18:48.252 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:18:48.252 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-12T18:18:48.252 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-12T18:18:48.252 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:18:48.484 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=2147483647 2026-02-12T18:18:48.485 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:18:48.485 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:18:48.485 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:18:48.485 INFO:tasks.workunit.client.0.trial116.stderr:6' 2026-02-12T18:18:48.485 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 2147483647 5 1 0 6 2026-02-12T18:18:48.485 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('2147483647' '5' '1' '0' '6') 2026-02-12T18:18:48.485 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-12T18:18:48.485 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=5 2026-02-12T18:18:48.485 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-12T18:18:48.485 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/5/type 2026-02-12T18:18:48.487 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-12T18:18:48.487 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 5 bluestore_debug_inject_read_err true 2026-02-12T18:18:48.487 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-12T18:18:48.488 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=5 2026-02-12T18:18:48.488 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-12T18:18:48.488 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-12T18:18:48.488 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-12T18:18:48.488 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.5 2026-02-12T18:18:48.488 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.5 2026-02-12T18:18:48.488 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.5 ']' 2026-02-12T18:18:48.488 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:18:48.488 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:18:48.488 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:18:48.488 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.5.asok 2026-02-12T18:18:48.488 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.76821/ceph-osd.5.asok config set bluestore_debug_inject_read_err true 2026-02-12T18:18:48.535 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-12T18:18:48.535 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-12T18:18:48.535 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-12T18:18:48.536 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.5 2026-02-12T18:18:48.536 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.5 2026-02-12T18:18:48.537 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.5 ']' 2026-02-12T18:18:48.537 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:18:48.537 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:18:48.537 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:18:48.537 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.5.asok 2026-02-12T18:18:48.537 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-12T18:18:48.537 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-osd.5.asok injectmdataerr pool-jerasure myobject 1 2026-02-12T18:18:48.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:398: TEST_ec_object_attr_read_error: activate_osd td/test-erasure-eio 3 2026-02-12T18:18:48.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-12T18:18:48.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T18:18:48.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-12T18:18:48.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T18:18:48.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-12T18:18:48.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:18:48.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:18:48.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:18:48.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:18:48.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-12T18:18:48.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-12T18:18:48.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T18:18:48.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T18:18:48.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:18:48.587 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T18:18:48.587 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:18:48.587 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:18:48.587 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:18:48.587 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:18:48.588 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:18:48.588 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:18:48.588 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:18:48.588 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:18:48.588 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:18:48.588 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:18:48.588 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:18:48.588 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:18:48.588 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:18:48.588 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:18:48.588 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T18:18:48.588 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-12T18:18:48.588 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-12T18:18:48.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-12T18:18:48.589 INFO:tasks.workunit.client.0.trial116.stdout:start osd.3 2026-02-12T18:18:48.590 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:18:48.590 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-12T18:18:48.590 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-12T18:18:48.590 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T18:18:48.590 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T18:18:48.590 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T18:18:48.599 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:48.596+0000 7fef36dd8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:48.601 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:48.597+0000 7fef36dd8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:48.602 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:48.598+0000 7fef36dd8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:48.740 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:48.736+0000 7fef36dd8780 -1 Falling back to public interface 2026-02-12T18:18:48.813 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-12T18:18:48.814 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:18:48.814 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-12T18:18:48.814 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:18:48.814 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:18:48.814 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:18:48.814 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:18:48.814 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:18:48.815 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:18:48.815 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:18:49.047 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:18:49.083 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:49.079+0000 7fef36dd8780 -1 osd.3 60 log_to_monitors true 2026-02-12T18:18:50.049 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:18:50.049 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:18:50.049 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:18:50.049 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:18:50.050 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:18:50.050 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:18:50.278 INFO:tasks.workunit.client.0.trial116.stdout:osd.3 up in weight 1 up_from 63 up_thru 63 down_at 60 last_clean_interval [27,60) [v2:127.0.0.1:6826/620978083,v1:127.0.0.1:6827/620978083] [v2:127.0.0.1:6828/620978083,v1:127.0.0.1:6829/620978083] exists,up d5e6f499-26c5-4213-ae9b-e703ab0c3efb 2026-02-12T18:18:50.279 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:18:50.280 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:18:50.280 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:18:50.280 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:401: TEST_ec_object_attr_read_error: wait_for_clean 2026-02-12T18:18:50.280 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:18:50.280 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:18:50.280 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:18:50.280 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:18:50.280 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:18:50.280 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:18:50.280 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:18:50.280 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:18:50.281 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:18:50.363 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:18:50.363 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:18:50.363 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:18:50.363 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:18:50.363 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:18:50.363 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:18:50.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:18:50.589 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:18:50.590 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:18:50.590 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:18:50.590 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:18:50.590 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:18:50.590 INFO:tasks.workunit.client.0.trial116.stderr:6' 2026-02-12T18:18:50.590 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:18:50.590 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:18:50.590 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:18:50.654 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803786 2026-02-12T18:18:50.654 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803786 2026-02-12T18:18:50.654 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786' 2026-02-12T18:18:50.654 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:18:50.655 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:18:50.717 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574857 2026-02-12T18:18:50.717 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574857 2026-02-12T18:18:50.717 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-55834574857' 2026-02-12T18:18:50.718 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:18:50.718 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:18:50.780 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411338 2026-02-12T18:18:50.781 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411338 2026-02-12T18:18:50.781 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-55834574857 2-77309411338' 2026-02-12T18:18:50.781 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:18:50.781 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:18:50.845 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=270582939651 2026-02-12T18:18:50.845 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 270582939651 2026-02-12T18:18:50.845 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-55834574857 2-77309411338 3-270582939651' 2026-02-12T18:18:50.845 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:18:50.845 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-12T18:18:50.910 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=154618822664 2026-02-12T18:18:50.910 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 154618822664 2026-02-12T18:18:50.911 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-55834574857 2-77309411338 3-270582939651 4-154618822664' 2026-02-12T18:18:50.911 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:18:50.911 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-12T18:18:50.974 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593735 2026-02-12T18:18:50.975 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593735 2026-02-12T18:18:50.975 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-55834574857 2-77309411338 3-270582939651 4-154618822664 5-184683593735' 2026-02-12T18:18:50.975 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:18:50.975 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-12T18:18:51.039 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=219043332104 2026-02-12T18:18:51.040 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 219043332104 2026-02-12T18:18:51.040 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803786 1-55834574857 2-77309411338 3-270582939651 4-154618822664 5-184683593735 6-219043332104' 2026-02-12T18:18:51.040 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:18:51.040 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803786 2026-02-12T18:18:51.040 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:18:51.041 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:18:51.041 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803786 2026-02-12T18:18:51.042 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:18:51.042 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803786 2026-02-12T18:18:51.043 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.0 seq 25769803786 2026-02-12T18:18:51.043 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803786' 2026-02-12T18:18:51.043 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:18:51.273 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803783 -lt 25769803786 2026-02-12T18:18:51.273 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:18:52.274 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:18:52.275 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:18:52.502 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803786 -lt 25769803786 2026-02-12T18:18:52.502 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:18:52.502 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574857 2026-02-12T18:18:52.503 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:18:52.503 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:18:52.504 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574857 2026-02-12T18:18:52.504 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:18:52.505 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574857 2026-02-12T18:18:52.506 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574857' 2026-02-12T18:18:52.506 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.1 seq 55834574857 2026-02-12T18:18:52.506 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:18:52.730 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574857 -lt 55834574857 2026-02-12T18:18:52.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:18:52.731 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411338 2026-02-12T18:18:52.731 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:18:52.732 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:18:52.733 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411338 2026-02-12T18:18:52.733 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:18:52.734 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411338 2026-02-12T18:18:52.734 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411338' 2026-02-12T18:18:52.734 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.2 seq 77309411338 2026-02-12T18:18:52.735 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:18:52.963 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411338 -lt 77309411338 2026-02-12T18:18:52.963 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:18:52.963 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-270582939651 2026-02-12T18:18:52.964 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:18:52.964 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:18:52.965 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-270582939651 2026-02-12T18:18:52.965 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:18:52.966 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=270582939651 2026-02-12T18:18:52.966 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 270582939651' 2026-02-12T18:18:52.967 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.3 seq 270582939651 2026-02-12T18:18:52.967 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:18:53.194 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 270582939651 -lt 270582939651 2026-02-12T18:18:53.194 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:18:53.195 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-154618822664 2026-02-12T18:18:53.195 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:18:53.196 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-12T18:18:53.196 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-154618822664 2026-02-12T18:18:53.197 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:18:53.197 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=154618822664 2026-02-12T18:18:53.198 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 154618822664' 2026-02-12T18:18:53.198 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.4 seq 154618822664 2026-02-12T18:18:53.198 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-12T18:18:53.428 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 154618822664 -lt 154618822664 2026-02-12T18:18:53.428 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:18:53.428 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-184683593735 2026-02-12T18:18:53.429 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:18:53.429 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-12T18:18:53.430 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-184683593735 2026-02-12T18:18:53.430 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:18:53.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593735 2026-02-12T18:18:53.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 184683593735' 2026-02-12T18:18:53.432 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.5 seq 184683593735 2026-02-12T18:18:53.432 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-12T18:18:53.661 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593735 -lt 184683593735 2026-02-12T18:18:53.661 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:18:53.661 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-219043332104 2026-02-12T18:18:53.661 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:18:53.662 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-12T18:18:53.663 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-219043332104 2026-02-12T18:18:53.663 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:18:53.663 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=219043332104 2026-02-12T18:18:53.664 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.6 seq 219043332104 2026-02-12T18:18:53.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 219043332104' 2026-02-12T18:18:53.664 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-12T18:18:53.890 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 219043332104 -lt 219043332104 2026-02-12T18:18:53.890 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:18:53.891 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:18:53.891 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:18:54.206 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:18:54.206 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:18:54.206 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:18:54.206 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:18:54.206 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:18:54.207 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:18:54.207 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:18:54.207 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:18:54.429 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:18:54.430 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:18:54.430 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:18:54.430 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:18:54.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:18:54.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:18:54.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:18:54.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:403: TEST_ec_object_attr_read_error: rados_get td/test-erasure-eio pool-jerasure myobject 2026-02-12T18:18:54.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-12T18:18:54.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-12T18:18:54.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=myobject 2026-02-12T18:18:54.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-12T18:18:54.750 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-12T18:18:54.750 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get myobject td/test-erasure-eio/COPY 2026-02-12T18:18:54.766 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-12T18:18:54.767 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-12T18:18:54.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:405: TEST_ec_object_attr_read_error: delete_erasure_coded_pool pool-jerasure 2026-02-12T18:18:54.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-12T18:18:54.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-12T18:18:55.037 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-jerasure' does not exist 2026-02-12T18:18:55.045 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-12T18:18:55.322 INFO:tasks.workunit.client.0.trial116.stderr:erasure-code-profile myprofile does not exist 2026-02-12T18:18:55.330 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-12T18:18:55.330 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-12T18:18:55.331 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:18:55.331 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-12T18:18:55.331 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:18:55.331 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:18:55.331 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:18:55.331 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:18:55.331 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:18:55.441 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:18:55.441 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:18:55.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:18:55.443 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:18:55.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:18:55.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:18:55.444 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:18:55.445 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:18:55.445 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:18:55.445 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:18:55.445 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:18:55.446 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:18:55.447 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:18:55.447 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-12T18:18:55.461 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:18:55.461 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:18:55.461 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:18:55.461 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76821 2026-02-12T18:18:55.464 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:18:55.464 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:18:55.464 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-12T18:18:55.464 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-12T18:18:55.464 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-12T18:18:55.464 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-12T18:18:55.464 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-12T18:18:55.464 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:18:55.464 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-12T18:18:55.464 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:18:55.464 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:18:55.465 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:18:55.465 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:18:55.465 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:18:55.465 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:18:55.465 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:18:55.466 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:18:55.466 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:18:55.467 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:18:55.467 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:18:55.468 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:18:55.468 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:18:55.468 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:18:55.469 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:18:55.469 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:18:55.469 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:18:55.470 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:18:55.470 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-12T18:18:55.472 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:18:55.472 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:18:55.472 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:18:55.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76821 2026-02-12T18:18:55.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:18:55.473 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:18:55.473 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-12T18:18:55.474 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T18:18:55.474 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:18:55.474 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:18:55.474 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76821 2026-02-12T18:18:55.475 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T18:18:55.476 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T18:18:55.476 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T18:18:55.476 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-12T18:18:55.477 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-12T18:18:55.477 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-12T18:18:55.477 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T18:18:55.477 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T18:18:55.477 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T18:18:55.477 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-12T18:18:55.477 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-12T18:18:55.494 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T18:18:55.494 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:18:55.495 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:18:55.495 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:18:55.495 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:18:55.495 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:18:55.495 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:18:55.495 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T18:18:55.517 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T18:18:55.518 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T18:18:55.518 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:18:55.518 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:18:55.519 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T18:18:55.519 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T18:18:55.520 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:18:55.520 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:18:55.520 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:18:55.520 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:18:55.520 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:18:55.520 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:18:55.520 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:18:55.520 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:18:55.520 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76821/ceph-mon.a.asok config get fsid 2026-02-12T18:18:55.562 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T18:18:55.563 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:18:55.563 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:18:55.563 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T18:18:55.563 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T18:18:55.563 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:18:55.564 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:18:55.564 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:18:55.564 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:18:55.564 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:18:55.564 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:18:55.564 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:18:55.565 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:18:55.565 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76821/ceph-mon.a.asok config get mon_host 2026-02-12T18:18:55.606 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-12T18:18:55.606 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-12T18:18:55.606 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T18:18:55.606 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T18:18:55.607 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T18:18:55.607 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-12T18:18:55.607 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T18:18:55.704 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T18:18:55.705 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:18:55.705 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:18:55.705 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:18:55.705 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:18:55.705 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:18:55.705 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:18:55.705 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:18:55.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:18:55.722 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-12T18:18:55.722 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T18:18:55.821 INFO:tasks.workunit.client.0.trial116.stderr:pool 'rbd' already exists 2026-02-12T18:18:55.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:18:56.831 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-12T18:18:56.831 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:18:56.831 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:18:56.831 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:18:56.831 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:18:56.831 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:18:56.831 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:18:56.831 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-12T18:18:56.832 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-mon.a.asok log flush 2026-02-12T18:18:56.870 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:18:56.874 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-12T18:18:56.875 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:18:55.507+0000 7f3da1c29c00 0 load: jerasure load: lrc 2026-02-12T18:18:56.876 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_multiple_errors td/test-erasure-eio 2026-02-12T18:18:56.876 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:438: TEST_ec_recovery_multiple_errors: local dir=td/test-erasure-eio 2026-02-12T18:18:56.876 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:439: TEST_ec_recovery_multiple_errors: local objname=myobject 2026-02-12T18:18:56.876 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:441: TEST_ec_recovery_multiple_errors: setup_osds 9 2026-02-12T18:18:56.877 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=9 2026-02-12T18:18:56.877 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-12T18:18:56.877 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 9 - 1 2026-02-12T18:18:56.881 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 8 2026-02-12T18:18:56.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:18:56.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-12T18:18:56.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:18:56.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:18:56.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T18:18:56.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:18:56.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-12T18:18:56.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:18:56.882 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:18:56.882 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:18:56.882 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:18:56.882 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-12T18:18:56.882 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-12T18:18:56.882 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:18:56.882 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:18:56.882 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:18:56.882 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:18:56.882 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:18:56.882 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:18:56.882 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:18:56.883 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:18:56.883 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:18:56.883 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:18:56.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:18:56.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:18:56.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:18:56.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:18:56.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:18:56.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:18:56.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:18:56.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:18:56.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:18:56.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:18:56.884 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:18:56.884 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:18:56.884 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-12T18:18:56.884 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:18:56.884 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=84c60313-5ede-4a81-a736-9dcc8eb6d1d0 2026-02-12T18:18:56.884 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 84c60313-5ede-4a81-a736-9dcc8eb6d1d0' 2026-02-12T18:18:56.884 INFO:tasks.workunit.client.0.trial116.stdout:add osd0 84c60313-5ede-4a81-a736-9dcc8eb6d1d0 2026-02-12T18:18:56.885 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:18:56.891 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCQGY5plaTuNBAAxzEpSIdmHjS47LLLavogVw== 2026-02-12T18:18:56.891 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCQGY5plaTuNBAAxzEpSIdmHjS47LLLavogVw=="}' 2026-02-12T18:18:56.891 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 84c60313-5ede-4a81-a736-9dcc8eb6d1d0 -i td/test-erasure-eio/0/new.json 2026-02-12T18:18:56.994 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:18:57.001 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-12T18:18:57.002 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCQGY5plaTuNBAAxzEpSIdmHjS47LLLavogVw== --osd-uuid 84c60313-5ede-4a81-a736-9dcc8eb6d1d0 2026-02-12T18:18:57.018 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:57.014+0000 7f46fc039780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:57.019 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:57.015+0000 7f46fc039780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:57.020 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:57.016+0000 7f46fc039780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:57.020 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:57.016+0000 7f46fc039780 -1 bdev(0x35b8c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-12T18:18:57.021 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:57.017+0000 7f46fc039780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-12T18:18:57.406 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-12T18:18:57.406 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:18:57.407 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T18:18:57.407 INFO:tasks.workunit.client.0.trial116.stdout:adding osd0 key to auth repository 2026-02-12T18:18:57.407 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:18:57.510 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T18:18:57.510 INFO:tasks.workunit.client.0.trial116.stdout:start osd.0 2026-02-12T18:18:57.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:18:57.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:18:57.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:18:57.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:18:57.524 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:57.520+0000 7f6d4a853780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:57.525 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:57.522+0000 7f6d4a853780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:57.526 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:57.523+0000 7f6d4a853780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:57.607 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T18:18:57.607 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:18:57.607 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T18:18:57.608 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:18:57.608 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:18:57.608 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:18:57.608 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:18:57.609 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:18:57.609 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:18:57.609 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:18:57.705 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:18:57.728 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:57.725+0000 7f6d4a853780 -1 Falling back to public interface 2026-02-12T18:18:58.024 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:58.020+0000 7f6d4a853780 -1 osd.0 0 log_to_monitors true 2026-02-12T18:18:58.707 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:18:58.707 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:18:58.707 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:18:58.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:18:58.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:18:58.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:18:58.934 INFO:tasks.workunit.client.0.trial116.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/545076651,v1:127.0.0.1:6803/545076651] [v2:127.0.0.1:6804/545076651,v1:127.0.0.1:6805/545076651] exists,up 84c60313-5ede-4a81-a736-9dcc8eb6d1d0 2026-02-12T18:18:58.938 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:18:58.938 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:18:58.938 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:18:58.938 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:18:58.938 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-12T18:18:58.939 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:18:58.939 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:18:58.939 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-12T18:18:58.939 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:18:58.939 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-12T18:18:58.939 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:18:58.939 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:18:58.939 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:18:58.939 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:18:58.939 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-12T18:18:58.939 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-12T18:18:58.939 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:18:58.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:18:58.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:18:58.940 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:18:58.940 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:18:58.940 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:18:58.940 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:18:58.940 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:18:58.940 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:18:58.940 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:18:58.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:18:58.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:18:58.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:18:58.941 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:18:58.941 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:18:58.941 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:18:58.941 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:18:58.941 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:18:58.941 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:18:58.941 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:18:58.941 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:18:58.941 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:18:58.941 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-12T18:18:58.941 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:18:58.941 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=62d826ff-819a-4b50-9c0e-a1e9a70fe8f4 2026-02-12T18:18:58.941 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 62d826ff-819a-4b50-9c0e-a1e9a70fe8f4' 2026-02-12T18:18:58.942 INFO:tasks.workunit.client.0.trial116.stdout:add osd1 62d826ff-819a-4b50-9c0e-a1e9a70fe8f4 2026-02-12T18:18:58.942 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:18:58.947 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCSGY5p5NNNOBAAi3Gne9PWSsrQMgAlOxPEJA== 2026-02-12T18:18:58.948 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCSGY5p5NNNOBAAi3Gne9PWSsrQMgAlOxPEJA=="}' 2026-02-12T18:18:58.948 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 62d826ff-819a-4b50-9c0e-a1e9a70fe8f4 -i td/test-erasure-eio/1/new.json 2026-02-12T18:18:59.177 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:18:59.185 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-12T18:18:59.185 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCSGY5p5NNNOBAAi3Gne9PWSsrQMgAlOxPEJA== --osd-uuid 62d826ff-819a-4b50-9c0e-a1e9a70fe8f4 2026-02-12T18:18:59.201 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:59.198+0000 7f15a0e0a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:59.202 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:59.199+0000 7f15a0e0a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:59.203 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:59.200+0000 7f15a0e0a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:59.204 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:59.200+0000 7f15a0e0a780 -1 bdev(0x2580c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-12T18:18:59.204 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:59.200+0000 7f15a0e0a780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-12T18:18:59.523 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-12T18:18:59.523 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:18:59.524 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-12T18:18:59.524 INFO:tasks.workunit.client.0.trial116.stdout:adding osd1 key to auth repository 2026-02-12T18:18:59.524 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:18:59.853 INFO:tasks.workunit.client.0.trial116.stdout:start osd.1 2026-02-12T18:18:59.854 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-12T18:18:59.854 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:18:59.854 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:18:59.854 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:18:59.854 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:18:59.868 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:59.864+0000 7f4fd9d3d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:59.870 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:59.866+0000 7f4fd9d3d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:18:59.870 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:18:59.867+0000 7f4fd9d3d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:00.062 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:00.059+0000 7f4fd9d3d780 -1 Falling back to public interface 2026-02-12T18:19:00.078 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-12T18:19:00.078 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:19:00.078 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T18:19:00.078 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:19:00.078 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:19:00.078 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:19:00.078 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:19:00.079 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:19:00.079 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:19:00.079 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:19:00.304 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:19:00.371 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:00.367+0000 7f4fd9d3d780 -1 osd.1 0 log_to_monitors true 2026-02-12T18:19:01.305 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:19:01.306 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:19:01.306 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:19:01.307 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:19:01.307 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:19:01.307 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:19:01.535 INFO:tasks.workunit.client.0.trial116.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/4176197803,v1:127.0.0.1:6811/4176197803] [v2:127.0.0.1:6812/4176197803,v1:127.0.0.1:6813/4176197803] exists,up 62d826ff-819a-4b50-9c0e-a1e9a70fe8f4 2026-02-12T18:19:01.539 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:19:01.539 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:19:01.539 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:19:01.539 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:19:01.539 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-12T18:19:01.539 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:19:01.540 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:19:01.540 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-12T18:19:01.540 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:19:01.540 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-12T18:19:01.540 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:19:01.540 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:19:01.540 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:19:01.540 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:19:01.540 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-12T18:19:01.540 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-12T18:19:01.540 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:19:01.541 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:19:01.541 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:19:01.541 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:19:01.541 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:19:01.541 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:19:01.541 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:19:01.541 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:19:01.541 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:19:01.541 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:19:01.541 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:19:01.541 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:19:01.541 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:19:01.541 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:19:01.542 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:19:01.542 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:19:01.542 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:19:01.542 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:19:01.542 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:19:01.542 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:19:01.542 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:19:01.542 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:19:01.542 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-12T18:19:01.542 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:19:01.542 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6a0f3ee6-1251-49eb-acb7-bab2533e9c79 2026-02-12T18:19:01.543 INFO:tasks.workunit.client.0.trial116.stdout:add osd2 6a0f3ee6-1251-49eb-acb7-bab2533e9c79 2026-02-12T18:19:01.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 6a0f3ee6-1251-49eb-acb7-bab2533e9c79' 2026-02-12T18:19:01.543 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:19:01.548 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCVGY5pWBmCIBAAIZ6gvZAtXqYKROmBvL+Cqw== 2026-02-12T18:19:01.548 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCVGY5pWBmCIBAAIZ6gvZAtXqYKROmBvL+Cqw=="}' 2026-02-12T18:19:01.549 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6a0f3ee6-1251-49eb-acb7-bab2533e9c79 -i td/test-erasure-eio/2/new.json 2026-02-12T18:19:01.786 INFO:tasks.workunit.client.0.trial116.stdout:2 2026-02-12T18:19:01.794 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-12T18:19:01.795 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCVGY5pWBmCIBAAIZ6gvZAtXqYKROmBvL+Cqw== --osd-uuid 6a0f3ee6-1251-49eb-acb7-bab2533e9c79 2026-02-12T18:19:01.811 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:01.807+0000 7f7afcc46780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:01.812 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:01.809+0000 7f7afcc46780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:01.813 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:01.810+0000 7f7afcc46780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:01.814 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:01.810+0000 7f7afcc46780 -1 bdev(0x3f9ec00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-12T18:19:01.814 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:01.810+0000 7f7afcc46780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-12T18:19:02.128 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-12T18:19:02.128 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:19:02.129 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-12T18:19:02.129 INFO:tasks.workunit.client.0.trial116.stdout:adding osd2 key to auth repository 2026-02-12T18:19:02.130 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:19:02.451 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-12T18:19:02.452 INFO:tasks.workunit.client.0.trial116.stdout:start osd.2 2026-02-12T18:19:02.453 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:19:02.453 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:19:02.453 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:19:02.453 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:19:02.466 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:02.462+0000 7fa9f6a29780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:02.467 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:02.464+0000 7fa9f6a29780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:02.468 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:02.465+0000 7fa9f6a29780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:02.602 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:02.598+0000 7fa9f6a29780 -1 Falling back to public interface 2026-02-12T18:19:02.675 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-12T18:19:02.675 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:19:02.675 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T18:19:02.675 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:19:02.675 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:19:02.676 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:19:02.676 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:19:02.676 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:19:02.676 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:19:02.676 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:19:02.894 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:02.890+0000 7fa9f6a29780 -1 osd.2 0 log_to_monitors true 2026-02-12T18:19:02.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:19:03.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:19:03.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:19:03.905 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:19:03.905 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:19:03.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:19:03.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:19:04.129 INFO:tasks.workunit.client.0.trial116.stdout:osd.2 up in weight 1 up_from 19 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2880158394,v1:127.0.0.1:6819/2880158394] [v2:127.0.0.1:6820/2880158394,v1:127.0.0.1:6821/2880158394] exists,up 6a0f3ee6-1251-49eb-acb7-bab2533e9c79 2026-02-12T18:19:04.132 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:19:04.133 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:19:04.133 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:19:04.133 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:19:04.133 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-12T18:19:04.133 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:19:04.133 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:19:04.133 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-12T18:19:04.133 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:19:04.133 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-12T18:19:04.133 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:19:04.134 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:19:04.134 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:19:04.134 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:19:04.134 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-12T18:19:04.134 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-12T18:19:04.134 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:19:04.134 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:19:04.134 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:19:04.134 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:19:04.134 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:19:04.134 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:19:04.134 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:19:04.135 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:19:04.135 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:19:04.135 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:19:04.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:19:04.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:19:04.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:19:04.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:19:04.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:19:04.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:19:04.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:19:04.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:19:04.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:19:04.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:19:04.136 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:19:04.136 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:19:04.136 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-12T18:19:04.136 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:19:04.136 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a443b4cd-075e-4d60-9c31-07599574e830 2026-02-12T18:19:04.136 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 a443b4cd-075e-4d60-9c31-07599574e830' 2026-02-12T18:19:04.136 INFO:tasks.workunit.client.0.trial116.stdout:add osd3 a443b4cd-075e-4d60-9c31-07599574e830 2026-02-12T18:19:04.137 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:19:04.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCYGY5plEFNCBAAgGX51HMP2bxJvpldpJxQXg== 2026-02-12T18:19:04.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCYGY5plEFNCBAAgGX51HMP2bxJvpldpJxQXg=="}' 2026-02-12T18:19:04.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a443b4cd-075e-4d60-9c31-07599574e830 -i td/test-erasure-eio/3/new.json 2026-02-12T18:19:04.375 INFO:tasks.workunit.client.0.trial116.stdout:3 2026-02-12T18:19:04.383 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-12T18:19:04.384 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCYGY5plEFNCBAAgGX51HMP2bxJvpldpJxQXg== --osd-uuid a443b4cd-075e-4d60-9c31-07599574e830 2026-02-12T18:19:04.400 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:04.396+0000 7f32eea5b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:04.401 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:04.397+0000 7f32eea5b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:04.402 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:04.398+0000 7f32eea5b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:04.402 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:04.399+0000 7f32eea5b780 -1 bdev(0x401ac00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-12T18:19:04.402 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:04.399+0000 7f32eea5b780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-12T18:19:04.714 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-12T18:19:04.715 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:19:04.715 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-12T18:19:04.716 INFO:tasks.workunit.client.0.trial116.stdout:adding osd3 key to auth repository 2026-02-12T18:19:04.716 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:19:05.038 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-12T18:19:05.038 INFO:tasks.workunit.client.0.trial116.stdout:start osd.3 2026-02-12T18:19:05.039 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:19:05.039 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:19:05.039 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:19:05.039 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:19:05.053 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:05.049+0000 7f04a4bd8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:05.055 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:05.051+0000 7f04a4bd8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:05.056 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:05.052+0000 7f04a4bd8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:05.192 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:05.189+0000 7f04a4bd8780 -1 Falling back to public interface 2026-02-12T18:19:05.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-12T18:19:05.264 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:19:05.264 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-12T18:19:05.264 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:19:05.264 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:19:05.264 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:19:05.264 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:19:05.265 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:19:05.265 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:19:05.265 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:19:05.487 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:19:05.501 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:05.497+0000 7f04a4bd8780 -1 osd.3 0 log_to_monitors true 2026-02-12T18:19:06.488 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:19:06.489 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:19:06.489 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:19:06.490 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:19:06.490 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:19:06.490 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:19:06.715 INFO:tasks.workunit.client.0.trial116.stdout:osd.3 up in weight 1 up_from 26 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2504142038,v1:127.0.0.1:6827/2504142038] [v2:127.0.0.1:6828/2504142038,v1:127.0.0.1:6829/2504142038] exists,up a443b4cd-075e-4d60-9c31-07599574e830 2026-02-12T18:19:06.718 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:19:06.718 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:19:06.718 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:19:06.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:19:06.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-12T18:19:06.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:19:06.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:19:06.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-12T18:19:06.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:19:06.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-12T18:19:06.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:19:06.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:19:06.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:19:06.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:19:06.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-12T18:19:06.720 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-12T18:19:06.720 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:19:06.720 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:19:06.720 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:19:06.720 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:19:06.720 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:19:06.720 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:19:06.720 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:19:06.720 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:19:06.720 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:19:06.720 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:19:06.720 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:19:06.721 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:19:06.721 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:19:06.721 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:19:06.721 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:19:06.721 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:19:06.721 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:19:06.721 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:19:06.721 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:19:06.721 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:19:06.721 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:19:06.721 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:19:06.721 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-12T18:19:06.721 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:19:06.722 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=caa3e0dd-93f2-4ae5-ab73-afc0fac3c734 2026-02-12T18:19:06.722 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 caa3e0dd-93f2-4ae5-ab73-afc0fac3c734' 2026-02-12T18:19:06.722 INFO:tasks.workunit.client.0.trial116.stdout:add osd4 caa3e0dd-93f2-4ae5-ab73-afc0fac3c734 2026-02-12T18:19:06.723 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:19:06.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCaGY5pK+JDKxAA3fxvpy1H+iPekq9fh5EeYQ== 2026-02-12T18:19:06.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCaGY5pK+JDKxAA3fxvpy1H+iPekq9fh5EeYQ=="}' 2026-02-12T18:19:06.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new caa3e0dd-93f2-4ae5-ab73-afc0fac3c734 -i td/test-erasure-eio/4/new.json 2026-02-12T18:19:06.957 INFO:tasks.workunit.client.0.trial116.stdout:4 2026-02-12T18:19:06.965 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-12T18:19:06.966 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCaGY5pK+JDKxAA3fxvpy1H+iPekq9fh5EeYQ== --osd-uuid caa3e0dd-93f2-4ae5-ab73-afc0fac3c734 2026-02-12T18:19:06.982 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:06.978+0000 7f878f04a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:06.983 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:06.979+0000 7f878f04a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:06.984 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:06.980+0000 7f878f04a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:06.985 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:06.981+0000 7f878f04a780 -1 bdev(0x2f1ac00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-12T18:19:06.985 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:06.981+0000 7f878f04a780 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-12T18:19:07.300 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-12T18:19:07.300 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:19:07.301 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-12T18:19:07.301 INFO:tasks.workunit.client.0.trial116.stdout:adding osd4 key to auth repository 2026-02-12T18:19:07.302 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:19:07.625 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-12T18:19:07.626 INFO:tasks.workunit.client.0.trial116.stdout:start osd.4 2026-02-12T18:19:07.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:19:07.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:19:07.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:19:07.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:19:07.640 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:07.636+0000 7fd8f962c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:07.641 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:07.637+0000 7fd8f962c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:07.641 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:07.638+0000 7fd8f962c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:07.785 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:07.781+0000 7fd8f962c780 -1 Falling back to public interface 2026-02-12T18:19:07.853 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-12T18:19:07.853 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:19:07.853 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-12T18:19:07.853 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:19:07.853 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:19:07.853 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:19:07.853 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:19:07.854 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:19:07.854 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:19:07.854 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-12T18:19:08.070 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:08.066+0000 7fd8f962c780 -1 osd.4 0 log_to_monitors true 2026-02-12T18:19:08.083 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:19:09.085 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:19:09.085 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:19:09.085 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:19:09.086 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:19:09.086 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:19:09.086 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-12T18:19:09.315 INFO:tasks.workunit.client.0.trial116.stdout:osd.4 up in weight 1 up_from 33 up_thru 34 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/2825872654,v1:127.0.0.1:6835/2825872654] [v2:127.0.0.1:6836/2825872654,v1:127.0.0.1:6837/2825872654] exists,up caa3e0dd-93f2-4ae5-ab73-afc0fac3c734 2026-02-12T18:19:09.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:19:09.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:19:09.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:19:09.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:19:09.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-12T18:19:09.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:19:09.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:19:09.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-12T18:19:09.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:19:09.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-12T18:19:09.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:19:09.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:19:09.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:19:09.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:19:09.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-12T18:19:09.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-12T18:19:09.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:19:09.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:19:09.322 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:19:09.322 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:19:09.322 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:19:09.322 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:19:09.322 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:19:09.322 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:19:09.322 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:19:09.322 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:19:09.322 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:19:09.322 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:19:09.322 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:19:09.322 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:19:09.322 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:19:09.323 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:19:09.323 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:19:09.323 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:19:09.323 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:19:09.323 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:19:09.323 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:19:09.323 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:19:09.323 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-12T18:19:09.323 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:19:09.323 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=516d4043-6ab5-429c-8eba-b96d252731bc 2026-02-12T18:19:09.323 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 516d4043-6ab5-429c-8eba-b96d252731bc' 2026-02-12T18:19:09.324 INFO:tasks.workunit.client.0.trial116.stdout:add osd5 516d4043-6ab5-429c-8eba-b96d252731bc 2026-02-12T18:19:09.324 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:19:09.329 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCdGY5pHbJyExAALG/YZK1iQy22RZU8yQ5DGw== 2026-02-12T18:19:09.329 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCdGY5pHbJyExAALG/YZK1iQy22RZU8yQ5DGw=="}' 2026-02-12T18:19:09.329 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 516d4043-6ab5-429c-8eba-b96d252731bc -i td/test-erasure-eio/5/new.json 2026-02-12T18:19:09.562 INFO:tasks.workunit.client.0.trial116.stdout:5 2026-02-12T18:19:09.570 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-12T18:19:09.571 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCdGY5pHbJyExAALG/YZK1iQy22RZU8yQ5DGw== --osd-uuid 516d4043-6ab5-429c-8eba-b96d252731bc 2026-02-12T18:19:09.586 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:09.583+0000 7f88a6e10780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:09.588 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:09.584+0000 7f88a6e10780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:09.589 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:09.585+0000 7f88a6e10780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:09.589 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:09.585+0000 7f88a6e10780 -1 bdev(0x290ec00 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-12T18:19:09.589 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:09.585+0000 7f88a6e10780 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-12T18:19:09.921 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-12T18:19:09.921 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:19:09.922 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-12T18:19:09.922 INFO:tasks.workunit.client.0.trial116.stdout:adding osd5 key to auth repository 2026-02-12T18:19:09.923 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:19:10.240 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-12T18:19:10.241 INFO:tasks.workunit.client.0.trial116.stdout:start osd.5 2026-02-12T18:19:10.242 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:19:10.242 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:19:10.242 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:19:10.242 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:19:10.255 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:10.251+0000 7effb6871780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:10.256 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:10.253+0000 7effb6871780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:10.257 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:10.254+0000 7effb6871780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:10.379 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:10.375+0000 7effb6871780 -1 Falling back to public interface 2026-02-12T18:19:10.471 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-12T18:19:10.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:19:10.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-12T18:19:10.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:19:10.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:19:10.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:19:10.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:19:10.472 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:19:10.473 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:19:10.473 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-12T18:19:10.699 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:10.695+0000 7effb6871780 -1 osd.5 0 log_to_monitors true 2026-02-12T18:19:10.713 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:19:11.714 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:19:11.715 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:19:11.715 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:19:11.716 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:19:11.716 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:19:11.716 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-12T18:19:11.940 INFO:tasks.workunit.client.0.trial116.stdout:osd.5 up in weight 1 up_from 42 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/2558988785,v1:127.0.0.1:6843/2558988785] [v2:127.0.0.1:6844/2558988785,v1:127.0.0.1:6845/2558988785] exists,up 516d4043-6ab5-429c-8eba-b96d252731bc 2026-02-12T18:19:11.944 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:19:11.945 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:19:11.945 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:19:11.945 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:19:11.945 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-12T18:19:11.945 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:19:11.945 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:19:11.945 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-12T18:19:11.945 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:19:11.945 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-12T18:19:11.945 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:19:11.945 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:19:11.946 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:19:11.946 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:19:11.946 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-12T18:19:11.946 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-12T18:19:11.946 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:19:11.946 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:19:11.946 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:19:11.946 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:19:11.946 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:19:11.946 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:19:11.946 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:19:11.946 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:19:11.947 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:19:11.947 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:19:11.947 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:19:11.947 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:19:11.947 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:19:11.947 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:19:11.947 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:19:11.947 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:19:11.947 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:19:11.947 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:19:11.947 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:19:11.947 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:19:11.947 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:19:11.948 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:19:11.948 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-12T18:19:11.948 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:19:11.948 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0f9467a4-9fd8-4e12-9d73-6c085ce1e45e 2026-02-12T18:19:11.948 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 0f9467a4-9fd8-4e12-9d73-6c085ce1e45e' 2026-02-12T18:19:11.948 INFO:tasks.workunit.client.0.trial116.stdout:add osd6 0f9467a4-9fd8-4e12-9d73-6c085ce1e45e 2026-02-12T18:19:11.949 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:19:11.953 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCfGY5pV42kOBAA3dAyFZJG3q2erOgaq3Bv/g== 2026-02-12T18:19:11.953 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCfGY5pV42kOBAA3dAyFZJG3q2erOgaq3Bv/g=="}' 2026-02-12T18:19:11.953 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0f9467a4-9fd8-4e12-9d73-6c085ce1e45e -i td/test-erasure-eio/6/new.json 2026-02-12T18:19:12.185 INFO:tasks.workunit.client.0.trial116.stdout:6 2026-02-12T18:19:12.194 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-12T18:19:12.194 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCfGY5pV42kOBAA3dAyFZJG3q2erOgaq3Bv/g== --osd-uuid 0f9467a4-9fd8-4e12-9d73-6c085ce1e45e 2026-02-12T18:19:12.210 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:12.207+0000 7f48f4630780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:12.212 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:12.208+0000 7f48f4630780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:12.213 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:12.209+0000 7f48f4630780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:12.213 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:12.209+0000 7f48f4630780 -1 bdev(0x3b66c00 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-12T18:19:12.213 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:12.210+0000 7f48f4630780 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-12T18:19:12.553 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-12T18:19:12.553 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:19:12.553 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-12T18:19:12.554 INFO:tasks.workunit.client.0.trial116.stdout:adding osd6 key to auth repository 2026-02-12T18:19:12.554 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:19:12.885 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-12T18:19:12.885 INFO:tasks.workunit.client.0.trial116.stdout:start osd.6 2026-02-12T18:19:12.886 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:19:12.886 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:19:12.886 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:19:12.886 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:19:12.899 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:12.895+0000 7f868d483780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:12.901 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:12.897+0000 7f868d483780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:12.901 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:12.898+0000 7f868d483780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:13.023 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:13.020+0000 7f868d483780 -1 Falling back to public interface 2026-02-12T18:19:13.112 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-12T18:19:13.112 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:19:13.112 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-12T18:19:13.112 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:19:13.112 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:19:13.113 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:19:13.113 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:19:13.113 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:19:13.113 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:19:13.114 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-12T18:19:13.344 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:19:13.377 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:13.374+0000 7f868d483780 -1 osd.6 0 log_to_monitors true 2026-02-12T18:19:14.345 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:19:14.345 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:19:14.346 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:19:14.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:19:14.347 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:19:14.347 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-12T18:19:14.581 INFO:tasks.workunit.client.0.trial116.stdout:osd.6 up in weight 1 up_from 50 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/205672200,v1:127.0.0.1:6851/205672200] [v2:127.0.0.1:6852/205672200,v1:127.0.0.1:6853/205672200] exists,up 0f9467a4-9fd8-4e12-9d73-6c085ce1e45e 2026-02-12T18:19:14.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:19:14.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:19:14.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:19:14.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:19:14.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 7 2026-02-12T18:19:14.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:19:14.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:19:14.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=7 2026-02-12T18:19:14.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:19:14.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/7 2026-02-12T18:19:14.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:19:14.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:19:14.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:19:14.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:19:14.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/7' 2026-02-12T18:19:14.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/7/journal' 2026-02-12T18:19:14.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:19:14.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:19:14.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:19:14.587 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:19:14.587 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:19:14.588 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:19:14.588 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:19:14.588 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:19:14.588 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:19:14.588 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:19:14.588 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:19:14.588 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:19:14.588 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:19:14.588 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:19:14.588 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:19:14.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:19:14.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:19:14.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:19:14.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:19:14.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:19:14.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:19:14.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:19:14.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/7 2026-02-12T18:19:14.589 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:19:14.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=596a9ab2-d240-4401-a3a3-6a5dcb73ada4 2026-02-12T18:19:14.590 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd7 596a9ab2-d240-4401-a3a3-6a5dcb73ada4' 2026-02-12T18:19:14.590 INFO:tasks.workunit.client.0.trial116.stdout:add osd7 596a9ab2-d240-4401-a3a3-6a5dcb73ada4 2026-02-12T18:19:14.590 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:19:14.595 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCiGY5pKm5OIxAAmVq0cV37demA6Tzh2dH82g== 2026-02-12T18:19:14.595 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCiGY5pKm5OIxAAmVq0cV37demA6Tzh2dH82g=="}' 2026-02-12T18:19:14.595 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 596a9ab2-d240-4401-a3a3-6a5dcb73ada4 -i td/test-erasure-eio/7/new.json 2026-02-12T18:19:14.828 INFO:tasks.workunit.client.0.trial116.stdout:7 2026-02-12T18:19:14.835 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/7/new.json 2026-02-12T18:19:14.836 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 7 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/7 --osd-journal=td/test-erasure-eio/7/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCiGY5pKm5OIxAAmVq0cV37demA6Tzh2dH82g== --osd-uuid 596a9ab2-d240-4401-a3a3-6a5dcb73ada4 2026-02-12T18:19:14.852 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:14.849+0000 7fbed462f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:14.853 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:14.850+0000 7fbed462f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:14.854 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:14.851+0000 7fbed462f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:14.854 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:14.851+0000 7fbed462f780 -1 bdev(0x38fcc00 td/test-erasure-eio/7/block) open stat got: (1) Operation not permitted 2026-02-12T18:19:14.854 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:14.852+0000 7fbed462f780 -1 bluestore(td/test-erasure-eio/7) _read_fsid unparsable uuid 2026-02-12T18:19:15.185 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/7/keyring 2026-02-12T18:19:15.185 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:19:15.185 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd7 key to auth repository 2026-02-12T18:19:15.186 INFO:tasks.workunit.client.0.trial116.stdout:adding osd7 key to auth repository 2026-02-12T18:19:15.186 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/7/keyring auth add osd.7 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:19:15.505 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.7 2026-02-12T18:19:15.506 INFO:tasks.workunit.client.0.trial116.stdout:start osd.7 2026-02-12T18:19:15.506 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 7 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/7 --osd-journal=td/test-erasure-eio/7/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:19:15.506 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:19:15.507 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:19:15.507 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:19:15.519 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:15.516+0000 7f8738855780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:15.521 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:15.518+0000 7f8738855780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:15.521 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:15.519+0000 7f8738855780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:15.733 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 7 2026-02-12T18:19:15.733 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:19:15.733 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=7 2026-02-12T18:19:15.733 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:19:15.733 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:19:15.733 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:19:15.734 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:19:15.734 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:19:15.734 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:15.729+0000 7f8738855780 -1 Falling back to public interface 2026-02-12T18:19:15.734 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:19:15.735 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-12T18:19:15.956 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:19:16.100 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:16.097+0000 7f8738855780 -1 osd.7 0 log_to_monitors true 2026-02-12T18:19:16.958 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:19:16.958 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:19:16.959 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:19:16.959 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:19:16.960 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:19:16.960 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-12T18:19:17.187 INFO:tasks.workunit.client.0.trial116.stdout:osd.7 up in weight 1 up_from 60 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6858/3706909423,v1:127.0.0.1:6859/3706909423] [v2:127.0.0.1:6860/3706909423,v1:127.0.0.1:6861/3706909423] exists,up 596a9ab2-d240-4401-a3a3-6a5dcb73ada4 2026-02-12T18:19:17.191 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:19:17.191 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:19:17.191 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:19:17.191 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:19:17.191 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 8 2026-02-12T18:19:17.191 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:19:17.191 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:19:17.191 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=8 2026-02-12T18:19:17.192 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:19:17.192 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/8 2026-02-12T18:19:17.192 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:19:17.192 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:19:17.192 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:19:17.192 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:19:17.192 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/8' 2026-02-12T18:19:17.192 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/8/journal' 2026-02-12T18:19:17.192 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:19:17.192 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:19:17.192 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:19:17.192 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:19:17.193 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:19:17.193 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:19:17.193 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:19:17.193 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:19:17.193 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:19:17.193 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:19:17.193 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:19:17.193 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:19:17.193 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:19:17.193 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:19:17.193 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:19:17.193 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:19:17.194 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:19:17.194 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:19:17.194 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:19:17.194 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:19:17.194 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:19:17.194 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:19:17.194 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/8 2026-02-12T18:19:17.194 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:19:17.194 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=db92d568-c43d-4083-a533-78f649f63e36 2026-02-12T18:19:17.194 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd8 db92d568-c43d-4083-a533-78f649f63e36' 2026-02-12T18:19:17.195 INFO:tasks.workunit.client.0.trial116.stdout:add osd8 db92d568-c43d-4083-a533-78f649f63e36 2026-02-12T18:19:17.195 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:19:17.200 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQClGY5pTQvDCxAA+roOzTup8+n1R7sbIkgkyQ== 2026-02-12T18:19:17.200 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQClGY5pTQvDCxAA+roOzTup8+n1R7sbIkgkyQ=="}' 2026-02-12T18:19:17.200 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new db92d568-c43d-4083-a533-78f649f63e36 -i td/test-erasure-eio/8/new.json 2026-02-12T18:19:17.435 INFO:tasks.workunit.client.0.trial116.stdout:8 2026-02-12T18:19:17.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/8/new.json 2026-02-12T18:19:17.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 8 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/8 --osd-journal=td/test-erasure-eio/8/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQClGY5pTQvDCxAA+roOzTup8+n1R7sbIkgkyQ== --osd-uuid db92d568-c43d-4083-a533-78f649f63e36 2026-02-12T18:19:17.460 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:17.457+0000 7faf7e03f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:17.462 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:17.459+0000 7faf7e03f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:17.462 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:17.460+0000 7faf7e03f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:17.463 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:17.460+0000 7faf7e03f780 -1 bdev(0x2492c00 td/test-erasure-eio/8/block) open stat got: (1) Operation not permitted 2026-02-12T18:19:17.463 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:17.460+0000 7faf7e03f780 -1 bluestore(td/test-erasure-eio/8) _read_fsid unparsable uuid 2026-02-12T18:19:17.779 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/8/keyring 2026-02-12T18:19:17.779 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:19:17.780 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd8 key to auth repository 2026-02-12T18:19:17.780 INFO:tasks.workunit.client.0.trial116.stdout:adding osd8 key to auth repository 2026-02-12T18:19:17.781 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/8/keyring auth add osd.8 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:19:18.098 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.8 2026-02-12T18:19:18.099 INFO:tasks.workunit.client.0.trial116.stdout:start osd.8 2026-02-12T18:19:18.100 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 8 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/8 --osd-journal=td/test-erasure-eio/8/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:19:18.100 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:19:18.100 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:19:18.100 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:19:18.113 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:18.110+0000 7fb1bee2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:18.114 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:18.112+0000 7fb1bee2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:18.115 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:18.113+0000 7fb1bee2d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:18.292 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:18.289+0000 7fb1bee2d780 -1 Falling back to public interface 2026-02-12T18:19:18.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 8 2026-02-12T18:19:18.327 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:19:18.327 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=8 2026-02-12T18:19:18.327 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:19:18.327 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:19:18.327 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:19:18.327 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:19:18.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:19:18.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:19:18.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-12T18:19:18.556 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:19:18.650 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:18.647+0000 7fb1bee2d780 -1 osd.8 0 log_to_monitors true 2026-02-12T18:19:19.557 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:19:19.558 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:19:19.558 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:19:19.559 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:19:19.559 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:19:19.559 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-12T18:19:19.783 INFO:tasks.workunit.client.0.trial116.stdout:osd.8 up in weight 1 up_from 70 up_thru 71 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6866/2235340653,v1:127.0.0.1:6867/2235340653] [v2:127.0.0.1:6868/2235340653,v1:127.0.0.1:6869/2235340653] exists,up db92d568-c43d-4083-a533-78f649f63e36 2026-02-12T18:19:19.786 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:19:19.786 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:19:19.786 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:19:19.786 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-12T18:19:19.786 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T18:19:19.786 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T18:19:19.786 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:19:19.787 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:19:19.787 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:19:19.787 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.0.asok 2026-02-12T18:19:19.787 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-12T18:19:19.787 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-osd.0.asok log flush 2026-02-12T18:19:19.828 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:19:19.832 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-12T18:19:19.835 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:18:57.727+0000 7f6d4a853780 0 load: jerasure load: lrc 2026-02-12T18:19:19.836 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:443: TEST_ec_recovery_multiple_errors: local poolname=pool-jerasure 2026-02-12T18:19:19.836 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:444: TEST_ec_recovery_multiple_errors: create_erasure_coded_pool pool-jerasure 4 4 2026-02-12T18:19:19.836 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-12T18:19:19.836 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-12T18:19:19.836 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=4 2026-02-12T18:19:19.837 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-12T18:19:19.837 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=4 2026-02-12T18:19:19.837 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-12T18:19:19.837 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=4 m=4 crush-failure-domain=osd 2026-02-12T18:19:20.125 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-12T18:19:20.125 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-12T18:19:20.499 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-jerasure' already exists 2026-02-12T18:19:20.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:19:21.510 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-12T18:19:21.510 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:19:21.510 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:19:21.510 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:19:21.510 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:19:21.511 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:19:21.511 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:19:21.511 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:19:21.512 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:19:21.512 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:19:21.595 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:19:21.596 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:19:21.596 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:19:21.596 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:19:21.596 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:19:21.596 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:19:21.824 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:19:21.824 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:19:21.824 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:19:21.825 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:19:21.825 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:19:21.825 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:19:21.825 INFO:tasks.workunit.client.0.trial116.stderr:6 2026-02-12T18:19:21.825 INFO:tasks.workunit.client.0.trial116.stderr:7 2026-02-12T18:19:21.825 INFO:tasks.workunit.client.0.trial116.stderr:8' 2026-02-12T18:19:21.825 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:19:21.825 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:19:21.825 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:19:21.891 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803784 2026-02-12T18:19:21.891 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803784 2026-02-12T18:19:21.891 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784' 2026-02-12T18:19:21.891 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:19:21.891 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:19:21.953 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574855 2026-02-12T18:19:21.954 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574855 2026-02-12T18:19:21.954 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-55834574855' 2026-02-12T18:19:21.954 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:19:21.954 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:19:22.019 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378631 2026-02-12T18:19:22.019 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378631 2026-02-12T18:19:22.020 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-55834574855 2-81604378631' 2026-02-12T18:19:22.020 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:19:22.020 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:19:22.082 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149703 2026-02-12T18:19:22.082 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149703 2026-02-12T18:19:22.082 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-55834574855 2-81604378631 3-111669149703' 2026-02-12T18:19:22.083 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:19:22.083 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-12T18:19:22.147 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920774 2026-02-12T18:19:22.147 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920774 2026-02-12T18:19:22.147 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-55834574855 2-81604378631 3-111669149703 4-141733920774' 2026-02-12T18:19:22.147 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:19:22.147 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-12T18:19:22.211 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626437 2026-02-12T18:19:22.211 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626437 2026-02-12T18:19:22.211 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-55834574855 2-81604378631 3-111669149703 4-141733920774 5-180388626437' 2026-02-12T18:19:22.212 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:19:22.212 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-12T18:19:22.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364804 2026-02-12T18:19:22.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364804 2026-02-12T18:19:22.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-55834574855 2-81604378631 3-111669149703 4-141733920774 5-180388626437 6-214748364804' 2026-02-12T18:19:22.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:19:22.277 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-12T18:19:22.340 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=257698037764 2026-02-12T18:19:22.341 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 257698037764 2026-02-12T18:19:22.341 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-55834574855 2-81604378631 3-111669149703 4-141733920774 5-180388626437 6-214748364804 7-257698037764' 2026-02-12T18:19:22.341 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:19:22.341 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-12T18:19:22.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=300647710723 2026-02-12T18:19:22.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 300647710723 2026-02-12T18:19:22.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-55834574855 2-81604378631 3-111669149703 4-141733920774 5-180388626437 6-214748364804 7-257698037764 8-300647710723' 2026-02-12T18:19:22.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:19:22.408 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803784 2026-02-12T18:19:22.408 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:19:22.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:19:22.409 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803784 2026-02-12T18:19:22.409 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:19:22.410 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803784 2026-02-12T18:19:22.410 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-12T18:19:22.411 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.0 seq 25769803784 2026-02-12T18:19:22.411 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:19:22.637 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803784 2026-02-12T18:19:22.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:19:23.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:19:23.639 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:19:23.862 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-12T18:19:23.863 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:19:23.864 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574855 2026-02-12T18:19:23.864 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:19:23.864 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:19:23.864 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574855 2026-02-12T18:19:23.865 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:19:23.865 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574855 2026-02-12T18:19:23.866 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574855' 2026-02-12T18:19:23.866 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.1 seq 55834574855 2026-02-12T18:19:23.866 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:19:24.092 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574855 -lt 55834574855 2026-02-12T18:19:24.093 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:19:24.093 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-81604378631 2026-02-12T18:19:24.094 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:19:24.094 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:19:24.094 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-81604378631 2026-02-12T18:19:24.095 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:19:24.096 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=81604378631 2026-02-12T18:19:24.096 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 81604378631' 2026-02-12T18:19:24.096 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.2 seq 81604378631 2026-02-12T18:19:24.096 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:19:24.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 81604378631 -lt 81604378631 2026-02-12T18:19:24.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:19:24.322 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149703 2026-02-12T18:19:24.322 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:19:24.322 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:19:24.323 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149703 2026-02-12T18:19:24.323 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:19:24.324 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149703 2026-02-12T18:19:24.324 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149703' 2026-02-12T18:19:24.325 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.3 seq 111669149703 2026-02-12T18:19:24.325 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:19:24.554 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149703 -lt 111669149703 2026-02-12T18:19:24.555 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:19:24.555 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-141733920774 2026-02-12T18:19:24.555 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:19:24.555 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-12T18:19:24.556 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-141733920774 2026-02-12T18:19:24.556 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:19:24.557 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=141733920774 2026-02-12T18:19:24.557 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 141733920774' 2026-02-12T18:19:24.558 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.4 seq 141733920774 2026-02-12T18:19:24.558 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-12T18:19:24.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 141733920774 -lt 141733920774 2026-02-12T18:19:24.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:19:24.782 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-180388626437 2026-02-12T18:19:24.782 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:19:24.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-12T18:19:24.783 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-180388626437 2026-02-12T18:19:24.783 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:19:24.784 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=180388626437 2026-02-12T18:19:24.785 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.5 seq 180388626437 2026-02-12T18:19:24.785 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 180388626437' 2026-02-12T18:19:24.785 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-12T18:19:25.012 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 180388626437 -lt 180388626437 2026-02-12T18:19:25.012 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:19:25.013 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-214748364804 2026-02-12T18:19:25.013 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:19:25.014 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-12T18:19:25.014 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-214748364804 2026-02-12T18:19:25.014 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:19:25.015 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=214748364804 2026-02-12T18:19:25.016 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 214748364804' 2026-02-12T18:19:25.016 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.6 seq 214748364804 2026-02-12T18:19:25.016 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-12T18:19:25.242 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 214748364804 -lt 214748364804 2026-02-12T18:19:25.242 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:19:25.243 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 7-257698037764 2026-02-12T18:19:25.243 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:19:25.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=7 2026-02-12T18:19:25.245 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-257698037764 2026-02-12T18:19:25.245 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:19:25.245 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=257698037764 2026-02-12T18:19:25.246 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.7 seq 257698037764 2026-02-12T18:19:25.246 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.7 seq 257698037764' 2026-02-12T18:19:25.246 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-12T18:19:25.470 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 257698037764 -lt 257698037764 2026-02-12T18:19:25.471 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:19:25.471 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-300647710723 2026-02-12T18:19:25.471 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:19:25.471 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-12T18:19:25.472 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-300647710723 2026-02-12T18:19:25.472 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:19:25.473 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=300647710723 2026-02-12T18:19:25.473 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 300647710723' 2026-02-12T18:19:25.473 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.8 seq 300647710723 2026-02-12T18:19:25.474 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-12T18:19:25.699 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 300647710723 -lt 300647710723 2026-02-12T18:19:25.699 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:19:25.699 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:19:25.699 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:19:26.011 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:19:26.011 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:19:26.011 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:19:26.011 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:19:26.011 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:19:26.011 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:19:26.012 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:19:26.012 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:19:26.238 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-12T18:19:26.238 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:19:26.239 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:19:26.239 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:19:26.574 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-12T18:19:26.574 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' -1 2026-02-12T18:19:26.574 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-12T18:19:26.574 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=4 2026-02-12T18:19:26.574 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-12T18:19:26.574 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-12T18:19:26.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-12T18:19:26.675 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:19:26.676 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:19:26.676 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:19:26.676 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:19:26.676 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:19:26.676 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:19:26.676 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:19:26.905 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-12T18:19:26.905 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:19:26.906 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:19:26.906 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:19:27.224 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-12T18:19:27.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-12T18:19:27.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-12T18:19:27.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-12T18:19:27.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-12T18:19:27.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-12T18:19:27.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-12T18:19:27.225 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-12T18:19:27.225 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-12T18:19:27.540 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-12T18:19:27.540 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-12T18:19:27.540 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 13 )) 2026-02-12T18:19:27.540 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-12T18:19:27.540 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-12T18:19:27.742 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-12T18:19:27.742 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:19:27.742 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:19:27.742 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:19:27.742 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:19:27.742 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:19:27.742 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:19:27.742 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:19:27.975 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:19:27.975 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:19:27.976 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:19:27.976 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:19:28.295 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:19:28.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:19:28.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:19:28.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:446: TEST_ec_recovery_multiple_errors: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-12T18:19:28.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-12T18:19:28.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-12T18:19:28.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-12T18:19:28.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:19:28.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-12T18:19:28.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:19:28.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-12T18:19:28.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:19:28.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-12T18:19:28.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:19:28.298 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-12T18:19:28.298 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-12T18:19:28.331 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:447: TEST_ec_recovery_multiple_errors: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 0 2026-02-12T18:19:28.331 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-12T18:19:28.331 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-12T18:19:28.331 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-12T18:19:28.332 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-12T18:19:28.332 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-12T18:19:28.332 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-12T18:19:28.332 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=myobject 2026-02-12T18:19:28.332 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-12T18:19:28.332 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-12T18:19:28.332 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-12T18:19:28.332 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=0 2026-02-12T18:19:28.332 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-12T18:19:28.332 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure myobject 2026-02-12T18:19:28.332 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:19:28.332 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-12T18:19:28.332 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-12T18:19:28.332 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:19:28.561 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:19:28.561 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:19:28.561 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:19:28.561 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:19:28.561 INFO:tasks.workunit.client.0.trial116.stderr:6 2026-02-12T18:19:28.561 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:19:28.562 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:19:28.562 INFO:tasks.workunit.client.0.trial116.stderr:7' 2026-02-12T18:19:28.562 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-12T18:19:28.562 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-12T18:19:28.562 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-12T18:19:28.562 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=3 2026-02-12T18:19:28.562 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-12T18:19:28.562 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/3/type 2026-02-12T18:19:28.564 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-12T18:19:28.564 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-12T18:19:28.564 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-12T18:19:28.564 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-12T18:19:28.564 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-12T18:19:28.565 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-12T18:19:28.565 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-12T18:19:28.565 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-12T18:19:28.565 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-12T18:19:28.565 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-12T18:19:28.565 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:19:28.565 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:19:28.565 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:19:28.565 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.3.asok 2026-02-12T18:19:28.565 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.76821/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-12T18:19:28.612 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-12T18:19:28.613 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-12T18:19:28.613 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-12T18:19:28.614 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.3 2026-02-12T18:19:28.614 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-12T18:19:28.614 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-12T18:19:28.615 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:19:28.615 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:19:28.615 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:19:28.615 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.3.asok 2026-02-12T18:19:28.615 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-12T18:19:28.615 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-osd.3.asok injectdataerr pool-jerasure myobject 0 2026-02-12T18:19:28.663 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:450: TEST_ec_recovery_multiple_errors: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 3 2026-02-12T18:19:28.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-12T18:19:28.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-12T18:19:28.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-12T18:19:28.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-12T18:19:28.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-12T18:19:28.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-12T18:19:28.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=myobject 2026-02-12T18:19:28.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-12T18:19:28.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-12T18:19:28.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-12T18:19:28.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=3 2026-02-12T18:19:28.665 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-12T18:19:28.665 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure myobject 2026-02-12T18:19:28.665 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:19:28.665 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-12T18:19:28.665 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-12T18:19:28.665 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:19:28.892 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:19:28.893 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:19:28.893 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:19:28.893 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:19:28.893 INFO:tasks.workunit.client.0.trial116.stderr:6 2026-02-12T18:19:28.893 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:19:28.893 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:19:28.893 INFO:tasks.workunit.client.0.trial116.stderr:7' 2026-02-12T18:19:28.893 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-12T18:19:28.893 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-12T18:19:28.893 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-12T18:19:28.894 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=0 2026-02-12T18:19:28.894 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-12T18:19:28.894 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/0/type 2026-02-12T18:19:28.895 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-12T18:19:28.896 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 0 bluestore_debug_inject_read_err true 2026-02-12T18:19:28.896 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-12T18:19:28.896 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=0 2026-02-12T18:19:28.896 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-12T18:19:28.896 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-12T18:19:28.896 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-12T18:19:28.896 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.0 2026-02-12T18:19:28.896 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T18:19:28.896 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T18:19:28.896 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:19:28.897 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:19:28.897 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:19:28.897 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.0.asok 2026-02-12T18:19:28.897 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.76821/ceph-osd.0.asok config set bluestore_debug_inject_read_err true 2026-02-12T18:19:28.942 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-12T18:19:28.942 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-12T18:19:28.944 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-12T18:19:28.944 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.0 2026-02-12T18:19:28.944 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T18:19:28.944 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T18:19:28.944 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:19:28.944 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:19:28.944 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:19:28.944 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.0.asok 2026-02-12T18:19:28.944 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-12T18:19:28.944 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-osd.0.asok injectdataerr pool-jerasure myobject 3 2026-02-12T18:19:28.994 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:451: TEST_ec_recovery_multiple_errors: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 4 2026-02-12T18:19:28.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-12T18:19:28.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-12T18:19:28.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-12T18:19:28.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-12T18:19:28.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-12T18:19:28.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-12T18:19:28.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=myobject 2026-02-12T18:19:28.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-12T18:19:28.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-12T18:19:28.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-12T18:19:28.996 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=4 2026-02-12T18:19:28.996 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-12T18:19:28.996 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure myobject 2026-02-12T18:19:28.996 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:19:28.996 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-12T18:19:28.996 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-12T18:19:28.996 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:19:29.226 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:19:29.226 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:19:29.226 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:19:29.226 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:19:29.226 INFO:tasks.workunit.client.0.trial116.stderr:6 2026-02-12T18:19:29.227 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:19:29.227 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:19:29.227 INFO:tasks.workunit.client.0.trial116.stderr:7' 2026-02-12T18:19:29.227 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-12T18:19:29.227 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-12T18:19:29.227 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-12T18:19:29.227 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=6 2026-02-12T18:19:29.227 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-12T18:19:29.227 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/6/type 2026-02-12T18:19:29.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-12T18:19:29.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 6 bluestore_debug_inject_read_err true 2026-02-12T18:19:29.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-12T18:19:29.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=6 2026-02-12T18:19:29.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-12T18:19:29.230 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-12T18:19:29.230 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-12T18:19:29.230 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.6 2026-02-12T18:19:29.230 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.6 2026-02-12T18:19:29.230 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.6 ']' 2026-02-12T18:19:29.230 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:19:29.230 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:19:29.230 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:19:29.230 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.6.asok 2026-02-12T18:19:29.230 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.76821/ceph-osd.6.asok config set bluestore_debug_inject_read_err true 2026-02-12T18:19:29.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-12T18:19:29.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-12T18:19:29.279 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-12T18:19:29.279 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.6 2026-02-12T18:19:29.279 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.6 2026-02-12T18:19:29.279 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.6 ']' 2026-02-12T18:19:29.279 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:19:29.279 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:19:29.280 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:19:29.280 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.6.asok 2026-02-12T18:19:29.280 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-12T18:19:29.280 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-osd.6.asok injectdataerr pool-jerasure myobject 4 2026-02-12T18:19:29.327 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:453: TEST_ec_recovery_multiple_errors: get_osds pool-jerasure myobject 2026-02-12T18:19:29.327 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:19:29.327 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-12T18:19:29.328 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-12T18:19:29.328 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:19:29.557 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:19:29.557 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:19:29.557 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:19:29.557 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:19:29.557 INFO:tasks.workunit.client.0.trial116.stderr:6 2026-02-12T18:19:29.557 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:19:29.558 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:19:29.558 INFO:tasks.workunit.client.0.trial116.stderr:7' 2026-02-12T18:19:29.558 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-12T18:19:29.558 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:453: TEST_ec_recovery_multiple_errors: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-12T18:19:29.558 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:453: TEST_ec_recovery_multiple_errors: local -a initial_osds 2026-02-12T18:19:29.558 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:454: TEST_ec_recovery_multiple_errors: local last_osd=7 2026-02-12T18:19:29.558 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:456: TEST_ec_recovery_multiple_errors: kill_daemons td/test-erasure-eio TERM osd.7 2026-02-12T18:19:29.558 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:19:29.558 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:19:29.558 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:19:29.558 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:19:29.558 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:19:29.863 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:19:29.864 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:457: TEST_ec_recovery_multiple_errors: ceph osd down 7 2026-02-12T18:19:30.091 INFO:tasks.workunit.client.0.trial116.stderr:osd.7 is already down. 2026-02-12T18:19:30.098 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:458: TEST_ec_recovery_multiple_errors: ceph osd out 7 2026-02-12T18:19:30.348 INFO:tasks.workunit.client.0.trial116.stderr:osd.7 is already out. 2026-02-12T18:19:30.356 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:461: TEST_ec_recovery_multiple_errors: wait_for_clean 2026-02-12T18:19:30.356 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:19:30.356 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:19:30.356 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:19:30.356 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:19:30.357 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:19:30.357 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:19:30.357 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:19:30.358 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:19:30.358 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:19:30.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:19:30.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:19:30.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:19:30.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:19:30.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:19:30.442 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:19:30.671 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:19:30.671 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:19:30.671 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:19:30.672 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:19:30.672 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:19:30.672 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:19:30.672 INFO:tasks.workunit.client.0.trial116.stderr:6 2026-02-12T18:19:30.672 INFO:tasks.workunit.client.0.trial116.stderr:7 2026-02-12T18:19:30.672 INFO:tasks.workunit.client.0.trial116.stderr:8' 2026-02-12T18:19:30.672 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:19:30.672 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:19:30.672 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:19:30.735 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803788 2026-02-12T18:19:30.735 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803788 2026-02-12T18:19:30.735 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788' 2026-02-12T18:19:30.736 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:19:30.736 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:19:30.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574858 2026-02-12T18:19:30.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574858 2026-02-12T18:19:30.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788 1-55834574858' 2026-02-12T18:19:30.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:19:30.802 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:19:30.866 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378635 2026-02-12T18:19:30.866 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378635 2026-02-12T18:19:30.867 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788 1-55834574858 2-81604378635' 2026-02-12T18:19:30.867 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:19:30.867 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:19:30.933 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149706 2026-02-12T18:19:30.933 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149706 2026-02-12T18:19:30.933 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788 1-55834574858 2-81604378635 3-111669149706' 2026-02-12T18:19:30.934 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:19:30.934 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-12T18:19:30.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920778 2026-02-12T18:19:30.996 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920778 2026-02-12T18:19:30.996 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788 1-55834574858 2-81604378635 3-111669149706 4-141733920778' 2026-02-12T18:19:30.996 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:19:30.996 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-12T18:19:31.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626440 2026-02-12T18:19:31.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626440 2026-02-12T18:19:31.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788 1-55834574858 2-81604378635 3-111669149706 4-141733920778 5-180388626440' 2026-02-12T18:19:31.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:19:31.062 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-12T18:19:31.124 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364808 2026-02-12T18:19:31.125 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364808 2026-02-12T18:19:31.125 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788 1-55834574858 2-81604378635 3-111669149706 4-141733920778 5-180388626440 6-214748364808' 2026-02-12T18:19:31.125 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:19:31.125 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-12T18:19:31.169 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-12T18:19:31.169 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: true 2026-02-12T18:19:31.169 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-12T18:19:31.169 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: continue 2026-02-12T18:19:31.170 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:19:31.170 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-12T18:19:31.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=300647710727 2026-02-12T18:19:31.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 300647710727 2026-02-12T18:19:31.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788 1-55834574858 2-81604378635 3-111669149706 4-141733920778 5-180388626440 6-214748364808 8-300647710727' 2026-02-12T18:19:31.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:19:31.235 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803788 2026-02-12T18:19:31.235 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:19:31.236 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:19:31.236 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803788 2026-02-12T18:19:31.236 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:19:31.237 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803788 2026-02-12T18:19:31.237 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803788' 2026-02-12T18:19:31.238 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.0 seq 25769803788 2026-02-12T18:19:31.238 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:19:31.471 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803786 -lt 25769803788 2026-02-12T18:19:31.471 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:19:32.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:19:32.472 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:19:32.700 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803788 -lt 25769803788 2026-02-12T18:19:32.700 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:19:32.701 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574858 2026-02-12T18:19:32.701 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:19:32.702 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:19:32.702 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574858 2026-02-12T18:19:32.702 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:19:32.703 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574858 2026-02-12T18:19:32.703 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574858' 2026-02-12T18:19:32.703 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.1 seq 55834574858 2026-02-12T18:19:32.704 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:19:32.935 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574859 -lt 55834574858 2026-02-12T18:19:32.936 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:19:32.936 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-81604378635 2026-02-12T18:19:32.936 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:19:32.937 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:19:32.938 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-81604378635 2026-02-12T18:19:32.938 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:19:32.939 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=81604378635 2026-02-12T18:19:32.939 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 81604378635' 2026-02-12T18:19:32.939 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.2 seq 81604378635 2026-02-12T18:19:32.940 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:19:33.168 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 81604378635 -lt 81604378635 2026-02-12T18:19:33.168 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:19:33.169 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149706 2026-02-12T18:19:33.169 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:19:33.169 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:19:33.170 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149706 2026-02-12T18:19:33.170 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:19:33.171 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149706 2026-02-12T18:19:33.171 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.3 seq 111669149706 2026-02-12T18:19:33.172 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149706' 2026-02-12T18:19:33.172 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:19:33.397 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149707 -lt 111669149706 2026-02-12T18:19:33.397 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:19:33.397 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-141733920778 2026-02-12T18:19:33.398 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:19:33.398 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-12T18:19:33.398 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-141733920778 2026-02-12T18:19:33.399 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:19:33.400 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=141733920778 2026-02-12T18:19:33.400 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 141733920778' 2026-02-12T18:19:33.401 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.4 seq 141733920778 2026-02-12T18:19:33.401 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-12T18:19:33.626 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 141733920778 -lt 141733920778 2026-02-12T18:19:33.626 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:19:33.627 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-180388626440 2026-02-12T18:19:33.627 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:19:33.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-12T18:19:33.628 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-180388626440 2026-02-12T18:19:33.628 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:19:33.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=180388626440 2026-02-12T18:19:33.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 180388626440' 2026-02-12T18:19:33.629 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.5 seq 180388626440 2026-02-12T18:19:33.630 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-12T18:19:33.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 180388626441 -lt 180388626440 2026-02-12T18:19:33.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:19:33.857 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-214748364808 2026-02-12T18:19:33.858 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:19:33.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-12T18:19:33.859 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-214748364808 2026-02-12T18:19:33.859 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:19:33.860 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=214748364808 2026-02-12T18:19:33.860 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 214748364808' 2026-02-12T18:19:33.860 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.6 seq 214748364808 2026-02-12T18:19:33.861 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-12T18:19:34.087 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 214748364808 -lt 214748364808 2026-02-12T18:19:34.088 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:19:34.088 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 8-300647710727 2026-02-12T18:19:34.088 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:19:34.089 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=8 2026-02-12T18:19:34.090 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-300647710727 2026-02-12T18:19:34.090 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:19:34.091 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=300647710727 2026-02-12T18:19:34.091 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.8 seq 300647710727' 2026-02-12T18:19:34.091 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.8 seq 300647710727 2026-02-12T18:19:34.091 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-12T18:19:34.318 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 300647710727 -lt 300647710727 2026-02-12T18:19:34.319 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:19:34.319 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:19:34.320 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:19:34.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:19:34.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:19:34.635 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:19:34.635 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:19:34.635 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:19:34.636 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:19:34.636 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:19:34.636 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:19:34.860 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:19:34.860 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:19:34.861 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:19:34.861 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:19:35.180 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:19:35.180 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:19:35.180 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:19:35.180 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:463: TEST_ec_recovery_multiple_errors: rados_get td/test-erasure-eio pool-jerasure myobject 2026-02-12T18:19:35.180 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-12T18:19:35.180 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-12T18:19:35.181 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=myobject 2026-02-12T18:19:35.181 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-12T18:19:35.181 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-12T18:19:35.181 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get myobject td/test-erasure-eio/COPY 2026-02-12T18:19:35.197 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-12T18:19:35.198 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-12T18:19:35.199 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:465: TEST_ec_recovery_multiple_errors: delete_erasure_coded_pool pool-jerasure 2026-02-12T18:19:35.199 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-12T18:19:35.199 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-12T18:19:35.469 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-jerasure' does not exist 2026-02-12T18:19:35.477 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-12T18:19:35.756 INFO:tasks.workunit.client.0.trial116.stderr:erasure-code-profile myprofile does not exist 2026-02-12T18:19:35.765 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-12T18:19:35.766 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-12T18:19:35.766 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:19:35.766 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-12T18:19:35.766 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:19:35.766 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:19:35.766 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:19:35.766 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:19:35.766 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:19:35.877 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:19:35.878 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:19:35.878 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:19:35.879 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:19:35.880 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:19:35.880 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:19:35.880 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:19:35.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:19:35.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:19:35.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:19:35.882 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:19:35.882 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:19:35.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:19:35.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-12T18:19:35.902 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:19:35.902 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:19:35.902 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:19:35.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76821 2026-02-12T18:19:35.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:19:35.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:19:35.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-12T18:19:35.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-12T18:19:35.905 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-12T18:19:35.905 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-12T18:19:35.905 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-12T18:19:35.905 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:19:35.905 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-12T18:19:35.905 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:19:35.905 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:19:35.905 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:19:35.905 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:19:35.905 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:19:35.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:19:35.906 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:19:35.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:19:35.907 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:19:35.908 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:19:35.908 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:19:35.908 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:19:35.909 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:19:35.909 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:19:35.910 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:19:35.910 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:19:35.910 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:19:35.911 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:19:35.911 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-12T18:19:35.912 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:19:35.913 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:19:35.913 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:19:35.913 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76821 2026-02-12T18:19:35.913 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:19:35.913 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:19:35.914 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-12T18:19:35.915 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T18:19:35.915 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:19:35.915 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:19:35.915 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76821 2026-02-12T18:19:35.916 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T18:19:35.916 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T18:19:35.916 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T18:19:35.916 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-12T18:19:35.916 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-12T18:19:35.916 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-12T18:19:35.916 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T18:19:35.916 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T18:19:35.917 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T18:19:35.917 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-12T18:19:35.917 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-12T18:19:35.934 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T18:19:35.935 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:19:35.935 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:19:35.935 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:19:35.935 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:19:35.935 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:19:35.935 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:19:35.935 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T18:19:35.959 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T18:19:35.960 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T18:19:35.960 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:19:35.960 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:19:35.960 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T18:19:35.960 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T18:19:35.960 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:19:35.960 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:19:35.960 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:19:35.961 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:19:35.961 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:19:35.961 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:19:35.961 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:19:35.961 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:19:35.961 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76821/ceph-mon.a.asok config get fsid 2026-02-12T18:19:36.004 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T18:19:36.004 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:19:36.004 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:19:36.004 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T18:19:36.004 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T18:19:36.006 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:19:36.006 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:19:36.006 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:19:36.006 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:19:36.006 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:19:36.006 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:19:36.006 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:19:36.006 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:19:36.006 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76821/ceph-mon.a.asok config get mon_host 2026-02-12T18:19:36.047 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-12T18:19:36.047 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-12T18:19:36.047 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T18:19:36.048 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T18:19:36.048 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T18:19:36.048 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-12T18:19:36.048 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T18:19:36.147 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T18:19:36.148 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:19:36.148 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:19:36.148 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:19:36.148 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:19:36.148 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:19:36.148 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:19:36.149 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:19:36.149 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:19:36.163 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-12T18:19:36.164 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T18:19:36.258 INFO:tasks.workunit.client.0.trial116.stderr:pool 'rbd' already exists 2026-02-12T18:19:36.264 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:19:37.267 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-12T18:19:37.267 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:19:37.267 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:19:37.267 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:19:37.267 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:19:37.267 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:19:37.267 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:19:37.268 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-12T18:19:37.268 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-mon.a.asok log flush 2026-02-12T18:19:37.306 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:19:37.311 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-12T18:19:37.312 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:19:35.949+0000 7f47ab44fc00 0 load: jerasure load: lrc 2026-02-12T18:19:37.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_multiple_objects td/test-erasure-eio 2026-02-12T18:19:37.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:471: TEST_ec_recovery_multiple_objects: local dir=td/test-erasure-eio 2026-02-12T18:19:37.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:472: TEST_ec_recovery_multiple_objects: local objname=myobject 2026-02-12T18:19:37.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:474: TEST_ec_recovery_multiple_objects: ORIG_ARGS='--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:19:37.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:475: TEST_ec_recovery_multiple_objects: CEPH_ARGS+=' --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-12T18:19:37.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:476: TEST_ec_recovery_multiple_objects: setup_osds 7 2026-02-12T18:19:37.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-12T18:19:37.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-12T18:19:37.315 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-12T18:19:37.318 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-12T18:19:37.319 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:19:37.319 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-12T18:19:37.319 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:19:37.319 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:19:37.319 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T18:19:37.319 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:19:37.319 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-12T18:19:37.319 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-12T18:19:37.319 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:19:37.319 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:19:37.319 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:19:37.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-12T18:19:37.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-12T18:19:37.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:19:37.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:19:37.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:19:37.320 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:19:37.320 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:19:37.320 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:19:37.320 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:19:37.320 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:19:37.320 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:19:37.320 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:19:37.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:19:37.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:19:37.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:19:37.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:19:37.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:19:37.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:19:37.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:19:37.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:19:37.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:19:37.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:19:37.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:19:37.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:19:37.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-12T18:19:37.322 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:19:37.322 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4f67e9d7-030e-4cc6-b60c-3e115332d107 2026-02-12T18:19:37.322 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 4f67e9d7-030e-4cc6-b60c-3e115332d107' 2026-02-12T18:19:37.322 INFO:tasks.workunit.client.0.trial116.stdout:add osd0 4f67e9d7-030e-4cc6-b60c-3e115332d107 2026-02-12T18:19:37.322 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:19:37.329 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC5GY5pf0tvExAAS42dTCwL4PenyNDz2as4JA== 2026-02-12T18:19:37.329 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC5GY5pf0tvExAAS42dTCwL4PenyNDz2as4JA=="}' 2026-02-12T18:19:37.329 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4f67e9d7-030e-4cc6-b60c-3e115332d107 -i td/test-erasure-eio/0/new.json 2026-02-12T18:19:37.434 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:19:37.440 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-12T18:19:37.441 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC5GY5pf0tvExAAS42dTCwL4PenyNDz2as4JA== --osd-uuid 4f67e9d7-030e-4cc6-b60c-3e115332d107 2026-02-12T18:19:37.460 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:37.456+0000 7f9323036780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:37.462 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:37.458+0000 7f9323036780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:37.463 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:37.459+0000 7f9323036780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:37.463 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:37.460+0000 7f9323036780 -1 bdev(0x287ac00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-12T18:19:37.464 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:37.460+0000 7f9323036780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-12T18:19:37.868 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-12T18:19:37.869 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:19:37.870 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T18:19:37.870 INFO:tasks.workunit.client.0.trial116.stdout:adding osd0 key to auth repository 2026-02-12T18:19:37.870 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:19:37.977 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T18:19:37.978 INFO:tasks.workunit.client.0.trial116.stdout:start osd.0 2026-02-12T18:19:37.979 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:19:37.979 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:19:37.979 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:19:37.979 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:19:37.994 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:37.990+0000 7f5935609780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:37.995 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:37.992+0000 7f5935609780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:37.996 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:37.993+0000 7f5935609780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:38.076 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T18:19:38.076 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:19:38.076 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T18:19:38.076 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:19:38.076 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:19:38.076 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:19:38.077 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:19:38.077 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:19:38.077 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:19:38.077 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:19:38.174 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:19:38.179 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:38.176+0000 7f5935609780 -1 Falling back to public interface 2026-02-12T18:19:38.460 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:38.456+0000 7f5935609780 -1 osd.0 0 log_to_monitors true 2026-02-12T18:19:39.176 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:19:39.176 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:19:39.176 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:19:39.177 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:19:39.177 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:19:39.177 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:19:39.406 INFO:tasks.workunit.client.0.trial116.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/93301895,v1:127.0.0.1:6803/93301895] [v2:127.0.0.1:6804/93301895,v1:127.0.0.1:6805/93301895] exists,up 4f67e9d7-030e-4cc6-b60c-3e115332d107 2026-02-12T18:19:39.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:19:39.410 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:19:39.410 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:19:39.410 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:19:39.410 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-12T18:19:39.410 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:19:39.410 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:19:39.410 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-12T18:19:39.410 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:19:39.411 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-12T18:19:39.411 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-12T18:19:39.411 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:19:39.411 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:19:39.411 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:19:39.411 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-12T18:19:39.411 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-12T18:19:39.412 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:19:39.412 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:19:39.412 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:19:39.412 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:19:39.412 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:19:39.412 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:19:39.412 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:19:39.412 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:19:39.412 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:19:39.412 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:19:39.412 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:19:39.412 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:19:39.413 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:19:39.413 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:19:39.413 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:19:39.413 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:19:39.413 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:19:39.413 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:19:39.413 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:19:39.413 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:19:39.413 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:19:39.413 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:19:39.413 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-12T18:19:39.413 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:19:39.414 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ea7ca9a9-e2f8-46d2-b02f-ddf1d80f52c2 2026-02-12T18:19:39.414 INFO:tasks.workunit.client.0.trial116.stdout:add osd1 ea7ca9a9-e2f8-46d2-b02f-ddf1d80f52c2 2026-02-12T18:19:39.414 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 ea7ca9a9-e2f8-46d2-b02f-ddf1d80f52c2' 2026-02-12T18:19:39.415 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:19:39.421 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC7GY5pcLrlGBAA7HTZ+8pqaLQ/gBS2vNmMVQ== 2026-02-12T18:19:39.421 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC7GY5pcLrlGBAA7HTZ+8pqaLQ/gBS2vNmMVQ=="}' 2026-02-12T18:19:39.421 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ea7ca9a9-e2f8-46d2-b02f-ddf1d80f52c2 -i td/test-erasure-eio/1/new.json 2026-02-12T18:19:39.652 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:19:39.660 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-12T18:19:39.660 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC7GY5pcLrlGBAA7HTZ+8pqaLQ/gBS2vNmMVQ== --osd-uuid ea7ca9a9-e2f8-46d2-b02f-ddf1d80f52c2 2026-02-12T18:19:39.678 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:39.674+0000 7f977f841780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:39.679 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:39.676+0000 7f977f841780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:39.680 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:39.677+0000 7f977f841780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:39.681 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:39.677+0000 7f977f841780 -1 bdev(0x3aeac00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-12T18:19:39.681 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:39.677+0000 7f977f841780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-12T18:19:40.008 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-12T18:19:40.008 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:19:40.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-12T18:19:40.009 INFO:tasks.workunit.client.0.trial116.stdout:adding osd1 key to auth repository 2026-02-12T18:19:40.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:19:40.327 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-12T18:19:40.327 INFO:tasks.workunit.client.0.trial116.stdout:start osd.1 2026-02-12T18:19:40.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:19:40.329 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:19:40.329 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:19:40.329 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:19:40.344 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:40.340+0000 7f907d53d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:40.345 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:40.342+0000 7f907d53d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:40.346 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:40.343+0000 7f907d53d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:40.476 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:40.473+0000 7f907d53d780 -1 Falling back to public interface 2026-02-12T18:19:40.556 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-12T18:19:40.556 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:19:40.556 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T18:19:40.556 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:19:40.556 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:19:40.556 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:19:40.557 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:19:40.557 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:19:40.558 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:19:40.558 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:19:40.766 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:40.762+0000 7f907d53d780 -1 osd.1 0 log_to_monitors true 2026-02-12T18:19:40.795 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:19:41.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:19:41.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:19:41.797 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:19:41.798 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:19:41.798 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:19:41.798 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:19:42.027 INFO:tasks.workunit.client.0.trial116.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3646908928,v1:127.0.0.1:6811/3646908928] [v2:127.0.0.1:6812/3646908928,v1:127.0.0.1:6813/3646908928] exists,up ea7ca9a9-e2f8-46d2-b02f-ddf1d80f52c2 2026-02-12T18:19:42.031 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:19:42.031 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:19:42.032 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:19:42.032 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:19:42.032 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-12T18:19:42.032 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:19:42.032 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:19:42.032 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-12T18:19:42.032 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:19:42.032 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-12T18:19:42.032 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-12T18:19:42.032 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:19:42.032 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:19:42.033 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:19:42.033 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-12T18:19:42.033 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-12T18:19:42.033 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:19:42.033 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:19:42.033 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:19:42.033 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:19:42.033 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:19:42.033 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:19:42.033 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:19:42.033 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:19:42.033 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:19:42.034 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:19:42.034 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:19:42.034 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:19:42.034 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:19:42.034 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:19:42.034 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:19:42.034 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:19:42.034 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:19:42.034 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:19:42.034 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:19:42.034 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:19:42.034 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:19:42.034 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:19:42.035 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-12T18:19:42.035 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:19:42.035 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1f0462e4-dd8e-40eb-a992-bfec716f3c7e 2026-02-12T18:19:42.035 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 1f0462e4-dd8e-40eb-a992-bfec716f3c7e' 2026-02-12T18:19:42.035 INFO:tasks.workunit.client.0.trial116.stdout:add osd2 1f0462e4-dd8e-40eb-a992-bfec716f3c7e 2026-02-12T18:19:42.035 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:19:42.043 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC+GY5pchRcAhAAz8NxK9dBJOimaSSQ48tKpw== 2026-02-12T18:19:42.043 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC+GY5pchRcAhAAz8NxK9dBJOimaSSQ48tKpw=="}' 2026-02-12T18:19:42.043 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1f0462e4-dd8e-40eb-a992-bfec716f3c7e -i td/test-erasure-eio/2/new.json 2026-02-12T18:19:42.280 INFO:tasks.workunit.client.0.trial116.stdout:2 2026-02-12T18:19:42.288 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-12T18:19:42.289 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC+GY5pchRcAhAAz8NxK9dBJOimaSSQ48tKpw== --osd-uuid 1f0462e4-dd8e-40eb-a992-bfec716f3c7e 2026-02-12T18:19:42.308 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:42.304+0000 7ff7e11d6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:42.309 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:42.305+0000 7ff7e11d6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:42.310 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:42.306+0000 7ff7e11d6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:42.310 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:42.306+0000 7ff7e11d6780 -1 bdev(0x28c0c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-12T18:19:42.310 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:42.307+0000 7ff7e11d6780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-12T18:19:42.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-12T18:19:42.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:19:42.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-12T18:19:42.621 INFO:tasks.workunit.client.0.trial116.stdout:adding osd2 key to auth repository 2026-02-12T18:19:42.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:19:42.942 INFO:tasks.workunit.client.0.trial116.stdout:start osd.2 2026-02-12T18:19:42.943 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-12T18:19:42.944 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:19:42.944 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:19:42.944 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:19:42.944 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:19:42.960 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:42.957+0000 7f99a361b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:42.962 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:42.958+0000 7f99a361b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:42.963 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:42.959+0000 7f99a361b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:43.084 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:43.081+0000 7f99a361b780 -1 Falling back to public interface 2026-02-12T18:19:43.172 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-12T18:19:43.173 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:19:43.173 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T18:19:43.173 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:19:43.173 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:19:43.173 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:19:43.173 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:19:43.173 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:19:43.174 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:19:43.174 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:19:43.403 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:19:43.465 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:43.461+0000 7f99a361b780 -1 osd.2 0 log_to_monitors true 2026-02-12T18:19:44.404 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:19:44.405 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:19:44.405 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:19:44.406 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:19:44.406 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:19:44.406 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:19:44.636 INFO:tasks.workunit.client.0.trial116.stdout:osd.2 up in weight 1 up_from 17 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/49111772,v1:127.0.0.1:6819/49111772] [v2:127.0.0.1:6820/49111772,v1:127.0.0.1:6821/49111772] exists,up 1f0462e4-dd8e-40eb-a992-bfec716f3c7e 2026-02-12T18:19:44.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:19:44.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:19:44.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:19:44.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:19:44.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-12T18:19:44.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:19:44.641 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:19:44.641 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-12T18:19:44.641 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:19:44.641 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-12T18:19:44.641 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-12T18:19:44.641 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:19:44.641 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:19:44.641 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:19:44.641 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-12T18:19:44.641 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-12T18:19:44.641 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:19:44.642 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:19:44.642 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:19:44.642 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:19:44.642 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:19:44.642 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:19:44.642 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:19:44.642 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:19:44.642 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:19:44.642 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:19:44.642 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:19:44.642 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:19:44.642 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:19:44.642 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:19:44.643 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:19:44.643 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:19:44.643 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:19:44.643 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:19:44.643 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:19:44.643 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:19:44.643 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:19:44.643 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:19:44.643 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-12T18:19:44.643 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:19:44.643 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=26bd647a-38ad-4221-aa68-719fb815ac6c 2026-02-12T18:19:44.643 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 26bd647a-38ad-4221-aa68-719fb815ac6c' 2026-02-12T18:19:44.644 INFO:tasks.workunit.client.0.trial116.stdout:add osd3 26bd647a-38ad-4221-aa68-719fb815ac6c 2026-02-12T18:19:44.644 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:19:44.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDAGY5p9NOaJhAABK8SkVM9ald4onx8BvIaLQ== 2026-02-12T18:19:44.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDAGY5p9NOaJhAABK8SkVM9ald4onx8BvIaLQ=="}' 2026-02-12T18:19:44.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 26bd647a-38ad-4221-aa68-719fb815ac6c -i td/test-erasure-eio/3/new.json 2026-02-12T18:19:44.885 INFO:tasks.workunit.client.0.trial116.stdout:3 2026-02-12T18:19:44.893 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-12T18:19:44.894 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDAGY5p9NOaJhAABK8SkVM9ald4onx8BvIaLQ== --osd-uuid 26bd647a-38ad-4221-aa68-719fb815ac6c 2026-02-12T18:19:44.912 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:44.908+0000 7ff458c2e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:44.913 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:44.909+0000 7ff458c2e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:44.914 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:44.910+0000 7ff458c2e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:44.914 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:44.910+0000 7ff458c2e780 -1 bdev(0x3c94c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-12T18:19:44.914 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:44.910+0000 7ff458c2e780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-12T18:19:45.233 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-12T18:19:45.233 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:19:45.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-12T18:19:45.234 INFO:tasks.workunit.client.0.trial116.stdout:adding osd3 key to auth repository 2026-02-12T18:19:45.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:19:45.576 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-12T18:19:45.576 INFO:tasks.workunit.client.0.trial116.stdout:start osd.3 2026-02-12T18:19:45.577 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:19:45.577 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:19:45.577 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:19:45.577 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:19:45.593 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:45.589+0000 7f962c013780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:45.594 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:45.590+0000 7f962c013780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:45.595 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:45.591+0000 7f962c013780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:45.721 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:45.717+0000 7f962c013780 -1 Falling back to public interface 2026-02-12T18:19:45.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-12T18:19:45.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:19:45.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-12T18:19:45.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:19:45.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:19:45.803 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:19:45.803 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:19:45.803 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:19:45.804 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:19:45.804 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:19:46.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:19:46.099 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:46.096+0000 7f962c013780 -1 osd.3 0 log_to_monitors true 2026-02-12T18:19:47.030 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:19:47.030 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:19:47.031 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:19:47.031 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:19:47.032 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:19:47.032 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:19:47.256 INFO:tasks.workunit.client.0.trial116.stdout:osd.3 up in weight 1 up_from 24 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/468691661,v1:127.0.0.1:6827/468691661] [v2:127.0.0.1:6828/468691661,v1:127.0.0.1:6829/468691661] exists,up 26bd647a-38ad-4221-aa68-719fb815ac6c 2026-02-12T18:19:47.260 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:19:47.260 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:19:47.260 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:19:47.260 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:19:47.260 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-12T18:19:47.260 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:19:47.260 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:19:47.260 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-12T18:19:47.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:19:47.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-12T18:19:47.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-12T18:19:47.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:19:47.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:19:47.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:19:47.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-12T18:19:47.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-12T18:19:47.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:19:47.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:19:47.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:19:47.261 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:19:47.262 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:19:47.262 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:19:47.262 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:19:47.262 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:19:47.262 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:19:47.262 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:19:47.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:19:47.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:19:47.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:19:47.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:19:47.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:19:47.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:19:47.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:19:47.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:19:47.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:19:47.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:19:47.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:19:47.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:19:47.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-12T18:19:47.263 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:19:47.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9c9401c6-ff31-41b1-abde-02101b3addc5 2026-02-12T18:19:47.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 9c9401c6-ff31-41b1-abde-02101b3addc5' 2026-02-12T18:19:47.264 INFO:tasks.workunit.client.0.trial116.stdout:add osd4 9c9401c6-ff31-41b1-abde-02101b3addc5 2026-02-12T18:19:47.264 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:19:47.271 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDDGY5pKev/DxAAhEPn+GrQuju54aCqeS8pkA== 2026-02-12T18:19:47.271 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDDGY5pKev/DxAAhEPn+GrQuju54aCqeS8pkA=="}' 2026-02-12T18:19:47.272 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9c9401c6-ff31-41b1-abde-02101b3addc5 -i td/test-erasure-eio/4/new.json 2026-02-12T18:19:47.517 INFO:tasks.workunit.client.0.trial116.stdout:4 2026-02-12T18:19:47.524 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-12T18:19:47.525 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDDGY5pKev/DxAAhEPn+GrQuju54aCqeS8pkA== --osd-uuid 9c9401c6-ff31-41b1-abde-02101b3addc5 2026-02-12T18:19:47.542 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:47.538+0000 7ff13c439780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:47.544 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:47.540+0000 7ff13c439780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:47.544 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:47.541+0000 7ff13c439780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:47.545 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:47.541+0000 7ff13c439780 -1 bdev(0x2a20c00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-12T18:19:47.545 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:47.541+0000 7ff13c439780 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-12T18:19:47.873 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-12T18:19:47.890 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:19:47.890 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-12T18:19:47.890 INFO:tasks.workunit.client.0.trial116.stdout:adding osd4 key to auth repository 2026-02-12T18:19:47.890 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:19:48.193 INFO:tasks.workunit.client.0.trial116.stdout:start osd.4 2026-02-12T18:19:48.194 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-12T18:19:48.194 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:19:48.194 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:19:48.194 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:19:48.194 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:19:48.209 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:48.205+0000 7fa6a1c12780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:48.210 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:48.207+0000 7fa6a1c12780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:48.211 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:48.208+0000 7fa6a1c12780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:48.348 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:48.344+0000 7fa6a1c12780 -1 Falling back to public interface 2026-02-12T18:19:48.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-12T18:19:48.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:19:48.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-12T18:19:48.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:19:48.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:19:48.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:19:48.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:19:48.431 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:19:48.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:19:48.432 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-12T18:19:48.659 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:19:48.669 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:48.665+0000 7fa6a1c12780 -1 osd.4 0 log_to_monitors true 2026-02-12T18:19:49.661 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:19:49.661 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:19:49.661 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:19:49.662 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:19:49.662 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:19:49.662 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-12T18:19:49.891 INFO:tasks.workunit.client.0.trial116.stdout:osd.4 up in weight 1 up_from 33 up_thru 34 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/2679554091,v1:127.0.0.1:6835/2679554091] [v2:127.0.0.1:6836/2679554091,v1:127.0.0.1:6837/2679554091] exists,up 9c9401c6-ff31-41b1-abde-02101b3addc5 2026-02-12T18:19:49.894 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:19:49.894 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:19:49.894 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:19:49.894 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:19:49.894 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-12T18:19:49.894 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:19:49.894 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:19:49.894 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-12T18:19:49.894 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:19:49.894 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-12T18:19:49.894 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-12T18:19:49.894 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:19:49.894 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:19:49.894 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:19:49.895 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-12T18:19:49.895 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-12T18:19:49.895 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:19:49.895 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:19:49.895 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:19:49.895 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:19:49.895 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:19:49.895 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:19:49.895 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:19:49.895 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:19:49.895 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:19:49.895 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:19:49.895 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:19:49.895 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:19:49.895 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:19:49.895 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:19:49.895 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:19:49.895 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:19:49.896 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:19:49.896 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:19:49.896 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:19:49.896 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:19:49.896 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:19:49.896 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:19:49.896 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-12T18:19:49.896 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:19:49.896 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a8b151a0-1104-4902-bc9b-d5c1a935b4c1 2026-02-12T18:19:49.896 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 a8b151a0-1104-4902-bc9b-d5c1a935b4c1' 2026-02-12T18:19:49.897 INFO:tasks.workunit.client.0.trial116.stdout:add osd5 a8b151a0-1104-4902-bc9b-d5c1a935b4c1 2026-02-12T18:19:49.897 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:19:49.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDFGY5p3inYNRAAU3GYZhKu01bMYwIlVBXeUA== 2026-02-12T18:19:49.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDFGY5p3inYNRAAU3GYZhKu01bMYwIlVBXeUA=="}' 2026-02-12T18:19:49.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a8b151a0-1104-4902-bc9b-d5c1a935b4c1 -i td/test-erasure-eio/5/new.json 2026-02-12T18:19:50.138 INFO:tasks.workunit.client.0.trial116.stdout:5 2026-02-12T18:19:50.147 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-12T18:19:50.148 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDFGY5p3inYNRAAU3GYZhKu01bMYwIlVBXeUA== --osd-uuid a8b151a0-1104-4902-bc9b-d5c1a935b4c1 2026-02-12T18:19:50.166 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:50.162+0000 7f737360d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:50.167 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:50.164+0000 7f737360d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:50.168 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:50.164+0000 7f737360d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:50.169 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:50.165+0000 7f737360d780 -1 bdev(0x4148c00 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-12T18:19:50.169 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:50.165+0000 7f737360d780 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-12T18:19:50.504 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-12T18:19:50.505 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:19:50.505 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-12T18:19:50.506 INFO:tasks.workunit.client.0.trial116.stdout:adding osd5 key to auth repository 2026-02-12T18:19:50.506 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:19:50.827 INFO:tasks.workunit.client.0.trial116.stdout:start osd.5 2026-02-12T18:19:50.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-12T18:19:50.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:19:50.829 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:19:50.829 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:19:50.829 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:19:50.844 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:50.841+0000 7fa9cf045780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:50.846 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:50.842+0000 7fa9cf045780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:50.846 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:50.843+0000 7fa9cf045780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:50.977 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:50.973+0000 7fa9cf045780 -1 Falling back to public interface 2026-02-12T18:19:51.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-12T18:19:51.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:19:51.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-12T18:19:51.060 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:19:51.060 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:19:51.060 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:19:51.060 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:19:51.060 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:19:51.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:19:51.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-12T18:19:51.289 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:51.285+0000 7fa9cf045780 -1 osd.5 0 log_to_monitors true 2026-02-12T18:19:51.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:19:52.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:19:52.298 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:19:52.298 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:19:52.298 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:19:52.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:19:52.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-12T18:19:52.528 INFO:tasks.workunit.client.0.trial116.stdout:osd.5 up in weight 1 up_from 41 up_thru 42 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/1506017662,v1:127.0.0.1:6843/1506017662] [v2:127.0.0.1:6844/1506017662,v1:127.0.0.1:6845/1506017662] exists,up a8b151a0-1104-4902-bc9b-d5c1a935b4c1 2026-02-12T18:19:52.532 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:19:52.532 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:19:52.532 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:19:52.532 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:19:52.532 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-12T18:19:52.532 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:19:52.532 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:19:52.532 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-12T18:19:52.532 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:19:52.533 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-12T18:19:52.533 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-12T18:19:52.533 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:19:52.533 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:19:52.533 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:19:52.533 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-12T18:19:52.533 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-12T18:19:52.533 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:19:52.533 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:19:52.533 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:19:52.533 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:19:52.533 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:19:52.534 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:19:52.534 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:19:52.534 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:19:52.534 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:19:52.534 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:19:52.534 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:19:52.534 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:19:52.534 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:19:52.534 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:19:52.534 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:19:52.534 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:19:52.534 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:19:52.535 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:19:52.535 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:19:52.535 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:19:52.535 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:19:52.535 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:19:52.535 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-12T18:19:52.535 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:19:52.535 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2771e173-d066-4003-aa6e-b7b5faad3a8f 2026-02-12T18:19:52.535 INFO:tasks.workunit.client.0.trial116.stdout:add osd6 2771e173-d066-4003-aa6e-b7b5faad3a8f 2026-02-12T18:19:52.536 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 2771e173-d066-4003-aa6e-b7b5faad3a8f' 2026-02-12T18:19:52.536 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:19:52.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDIGY5pq4YxIBAAgCTuF6Ve/dEe5uGuDw1BuA== 2026-02-12T18:19:52.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDIGY5pq4YxIBAAgCTuF6Ve/dEe5uGuDw1BuA=="}' 2026-02-12T18:19:52.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2771e173-d066-4003-aa6e-b7b5faad3a8f -i td/test-erasure-eio/6/new.json 2026-02-12T18:19:52.772 INFO:tasks.workunit.client.0.trial116.stdout:6 2026-02-12T18:19:52.779 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-12T18:19:52.781 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDIGY5pq4YxIBAAgCTuF6Ve/dEe5uGuDw1BuA== --osd-uuid 2771e173-d066-4003-aa6e-b7b5faad3a8f 2026-02-12T18:19:52.799 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:52.796+0000 7fa156433780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:52.801 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:52.797+0000 7fa156433780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:52.802 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:52.798+0000 7fa156433780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:52.802 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:52.798+0000 7fa156433780 -1 bdev(0x2aaac00 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-12T18:19:52.802 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:52.798+0000 7fa156433780 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-12T18:19:53.153 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-12T18:19:53.154 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:19:53.155 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-12T18:19:53.155 INFO:tasks.workunit.client.0.trial116.stdout:adding osd6 key to auth repository 2026-02-12T18:19:53.155 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:19:53.485 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-12T18:19:53.486 INFO:tasks.workunit.client.0.trial116.stdout:start osd.6 2026-02-12T18:19:53.486 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:19:53.487 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:19:53.487 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:19:53.487 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:19:53.503 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:53.499+0000 7f0714808780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:53.504 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:53.501+0000 7f0714808780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:53.505 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:53.502+0000 7f0714808780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:19:53.629 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:53.626+0000 7f0714808780 -1 Falling back to public interface 2026-02-12T18:19:53.714 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-12T18:19:53.714 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:19:53.714 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-12T18:19:53.714 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:19:53.714 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:19:53.714 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:19:53.714 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:19:53.715 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:19:53.715 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:19:53.715 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-12T18:19:53.949 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:19:53.979 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:19:53.976+0000 7f0714808780 -1 osd.6 0 log_to_monitors true 2026-02-12T18:19:54.951 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:19:54.951 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:19:54.951 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:19:54.952 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:19:54.952 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:19:54.952 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-12T18:19:55.194 INFO:tasks.workunit.client.0.trial116.stdout:osd.6 up in weight 1 up_from 49 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/3748974383,v1:127.0.0.1:6851/3748974383] [v2:127.0.0.1:6852/3748974383,v1:127.0.0.1:6853/3748974383] exists,up 2771e173-d066-4003-aa6e-b7b5faad3a8f 2026-02-12T18:19:55.196 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:19:55.196 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:19:55.196 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:19:55.196 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-12T18:19:55.196 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T18:19:55.196 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T18:19:55.196 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:19:55.196 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:19:55.196 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:19:55.197 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.0.asok 2026-02-12T18:19:55.197 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-12T18:19:55.197 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-osd.0.asok log flush 2026-02-12T18:19:55.242 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:19:55.246 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-12T18:19:55.248 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:19:38.178+0000 7f5935609780 0 load: jerasure load: lrc 2026-02-12T18:19:55.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:477: TEST_ec_recovery_multiple_objects: CEPH_ARGS='--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:19:55.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:479: TEST_ec_recovery_multiple_objects: local poolname=pool-jerasure 2026-02-12T18:19:55.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:480: TEST_ec_recovery_multiple_objects: create_erasure_coded_pool pool-jerasure 3 2 2026-02-12T18:19:55.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-12T18:19:55.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-12T18:19:55.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-12T18:19:55.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-12T18:19:55.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-12T18:19:55.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-12T18:19:55.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-12T18:19:55.536 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-12T18:19:55.536 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-12T18:19:55.913 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-jerasure' already exists 2026-02-12T18:19:55.921 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:19:56.923 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-12T18:19:56.923 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:19:56.923 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:19:56.923 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:19:56.924 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:19:56.924 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:19:56.925 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:19:56.925 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:19:56.925 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:19:56.925 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:19:57.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:19:57.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:19:57.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:19:57.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:19:57.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:19:57.010 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:19:57.240 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:19:57.241 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:19:57.241 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:19:57.241 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:19:57.241 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:19:57.241 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:19:57.241 INFO:tasks.workunit.client.0.trial116.stderr:6' 2026-02-12T18:19:57.241 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:19:57.241 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:19:57.241 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:19:57.306 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803782 2026-02-12T18:19:57.306 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803782 2026-02-12T18:19:57.306 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782' 2026-02-12T18:19:57.306 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:19:57.306 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:19:57.370 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607558 2026-02-12T18:19:57.370 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607558 2026-02-12T18:19:57.370 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607558' 2026-02-12T18:19:57.370 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:19:57.370 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:19:57.435 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444037 2026-02-12T18:19:57.435 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444037 2026-02-12T18:19:57.435 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-73014444037' 2026-02-12T18:19:57.436 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:19:57.436 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:19:57.500 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=103079215109 2026-02-12T18:19:57.500 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 103079215109 2026-02-12T18:19:57.500 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-73014444037 3-103079215109' 2026-02-12T18:19:57.501 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:19:57.501 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-12T18:19:57.565 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920772 2026-02-12T18:19:57.565 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920772 2026-02-12T18:19:57.565 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-73014444037 3-103079215109 4-141733920772' 2026-02-12T18:19:57.565 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:19:57.565 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-12T18:19:57.630 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=176093659141 2026-02-12T18:19:57.630 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 176093659141 2026-02-12T18:19:57.630 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-73014444037 3-103079215109 4-141733920772 5-176093659141' 2026-02-12T18:19:57.631 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:19:57.631 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-12T18:19:57.693 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397507 2026-02-12T18:19:57.694 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397507 2026-02-12T18:19:57.694 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-73014444037 3-103079215109 4-141733920772 5-176093659141 6-210453397507' 2026-02-12T18:19:57.694 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:19:57.694 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803782 2026-02-12T18:19:57.695 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:19:57.695 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:19:57.696 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803782 2026-02-12T18:19:57.696 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:19:57.697 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803782 2026-02-12T18:19:57.697 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803782' 2026-02-12T18:19:57.697 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.0 seq 25769803782 2026-02-12T18:19:57.697 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:19:57.923 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803780 -lt 25769803782 2026-02-12T18:19:57.923 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:19:58.924 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:19:58.925 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:19:59.150 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803782 2026-02-12T18:19:59.151 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:19:59.151 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607558 2026-02-12T18:19:59.151 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:19:59.152 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:19:59.153 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607558 2026-02-12T18:19:59.153 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:19:59.154 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607558 2026-02-12T18:19:59.154 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607558' 2026-02-12T18:19:59.154 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.1 seq 51539607558 2026-02-12T18:19:59.155 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:19:59.383 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607558 -lt 51539607558 2026-02-12T18:19:59.384 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:19:59.384 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-73014444037 2026-02-12T18:19:59.384 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:19:59.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:19:59.386 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444037 2026-02-12T18:19:59.386 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:19:59.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=73014444037 2026-02-12T18:19:59.387 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.2 seq 73014444037 2026-02-12T18:19:59.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 73014444037' 2026-02-12T18:19:59.388 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:19:59.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 73014444037 -lt 73014444037 2026-02-12T18:19:59.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:19:59.621 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-103079215109 2026-02-12T18:19:59.621 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:19:59.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:19:59.622 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-103079215109 2026-02-12T18:19:59.622 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:19:59.623 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=103079215109 2026-02-12T18:19:59.623 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 103079215109' 2026-02-12T18:19:59.623 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.3 seq 103079215109 2026-02-12T18:19:59.624 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:19:59.852 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 103079215109 -lt 103079215109 2026-02-12T18:19:59.852 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:19:59.852 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-141733920772 2026-02-12T18:19:59.852 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:19:59.853 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-12T18:19:59.854 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-141733920772 2026-02-12T18:19:59.854 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:19:59.855 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=141733920772 2026-02-12T18:19:59.855 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 141733920772' 2026-02-12T18:19:59.855 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.4 seq 141733920772 2026-02-12T18:19:59.856 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-12T18:20:00.083 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 141733920772 -lt 141733920772 2026-02-12T18:20:00.083 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:20:00.083 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-176093659141 2026-02-12T18:20:00.084 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:20:00.084 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-12T18:20:00.085 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-176093659141 2026-02-12T18:20:00.085 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:20:00.086 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=176093659141 2026-02-12T18:20:00.086 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 176093659141' 2026-02-12T18:20:00.086 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.5 seq 176093659141 2026-02-12T18:20:00.087 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-12T18:20:00.315 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 176093659141 -lt 176093659141 2026-02-12T18:20:00.315 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:20:00.315 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-210453397507 2026-02-12T18:20:00.316 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:20:00.316 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-12T18:20:00.317 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-210453397507 2026-02-12T18:20:00.317 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:20:00.318 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=210453397507 2026-02-12T18:20:00.318 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 210453397507' 2026-02-12T18:20:00.318 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.6 seq 210453397507 2026-02-12T18:20:00.319 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-12T18:20:00.544 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 210453397508 -lt 210453397507 2026-02-12T18:20:00.545 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:20:00.545 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:20:00.545 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:20:00.862 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:20:00.862 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:20:00.863 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:20:00.863 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:20:00.863 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:20:00.863 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:20:00.863 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:20:00.863 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:20:01.085 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:20:01.086 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:20:01.086 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:20:01.086 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:20:01.403 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:20:01.403 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:20:01.403 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:20:01.403 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:482: TEST_ec_recovery_multiple_objects: rados_put td/test-erasure-eio pool-jerasure test1 2026-02-12T18:20:01.403 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-12T18:20:01.403 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-12T18:20:01.404 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test1 2026-02-12T18:20:01.404 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:20:01.404 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-12T18:20:01.404 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:20:01.404 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-12T18:20:01.404 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:20:01.404 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-12T18:20:01.404 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:20:01.404 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-12T18:20:01.404 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test1 td/test-erasure-eio/ORIGINAL 2026-02-12T18:20:01.424 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:483: TEST_ec_recovery_multiple_objects: rados_put td/test-erasure-eio pool-jerasure test2 2026-02-12T18:20:01.424 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-12T18:20:01.424 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-12T18:20:01.425 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test2 2026-02-12T18:20:01.425 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:20:01.425 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-12T18:20:01.425 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:20:01.425 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-12T18:20:01.425 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:20:01.425 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-12T18:20:01.425 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:20:01.425 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-12T18:20:01.425 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test2 td/test-erasure-eio/ORIGINAL 2026-02-12T18:20:01.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:484: TEST_ec_recovery_multiple_objects: rados_put td/test-erasure-eio pool-jerasure test3 2026-02-12T18:20:01.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-12T18:20:01.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-12T18:20:01.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test3 2026-02-12T18:20:01.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:20:01.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-12T18:20:01.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:20:01.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-12T18:20:01.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:20:01.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-12T18:20:01.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:20:01.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-12T18:20:01.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test3 td/test-erasure-eio/ORIGINAL 2026-02-12T18:20:01.459 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:486: TEST_ec_recovery_multiple_objects: ceph osd out 0 2026-02-12T18:20:01.738 INFO:tasks.workunit.client.0.trial116.stderr:osd.0 is already out. 2026-02-12T18:20:01.747 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:489: TEST_ec_recovery_multiple_objects: wait_for_clean 2026-02-12T18:20:01.747 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:20:01.747 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:20:01.747 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:20:01.747 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:20:01.748 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:20:01.748 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:20:01.748 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:20:01.748 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:20:01.748 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:20:01.832 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:20:01.832 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:20:01.832 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:20:01.832 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:20:01.832 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:20:01.832 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:20:02.067 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:20:02.067 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:20:02.067 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:20:02.067 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:20:02.067 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:20:02.067 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:20:02.068 INFO:tasks.workunit.client.0.trial116.stderr:6' 2026-02-12T18:20:02.068 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:20:02.068 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:20:02.068 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:20:02.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803785 2026-02-12T18:20:02.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803785 2026-02-12T18:20:02.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785' 2026-02-12T18:20:02.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:20:02.135 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:20:02.198 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607561 2026-02-12T18:20:02.199 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607561 2026-02-12T18:20:02.199 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607561' 2026-02-12T18:20:02.199 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:20:02.199 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:20:02.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444040 2026-02-12T18:20:02.264 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444040 2026-02-12T18:20:02.264 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607561 2-73014444040' 2026-02-12T18:20:02.264 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:20:02.264 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:20:02.331 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=103079215112 2026-02-12T18:20:02.331 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 103079215112 2026-02-12T18:20:02.331 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607561 2-73014444040 3-103079215112' 2026-02-12T18:20:02.331 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:20:02.331 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-12T18:20:02.398 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920775 2026-02-12T18:20:02.398 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920775 2026-02-12T18:20:02.398 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607561 2-73014444040 3-103079215112 4-141733920775' 2026-02-12T18:20:02.399 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:20:02.399 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-12T18:20:02.461 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=176093659144 2026-02-12T18:20:02.461 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 176093659144 2026-02-12T18:20:02.461 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607561 2-73014444040 3-103079215112 4-141733920775 5-176093659144' 2026-02-12T18:20:02.462 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:20:02.462 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-12T18:20:02.526 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397510 2026-02-12T18:20:02.526 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397510 2026-02-12T18:20:02.526 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-51539607561 2-73014444040 3-103079215112 4-141733920775 5-176093659144 6-210453397510' 2026-02-12T18:20:02.526 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:20:02.527 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803785 2026-02-12T18:20:02.527 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:20:02.527 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:20:02.528 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803785 2026-02-12T18:20:02.528 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:20:02.529 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803785 2026-02-12T18:20:02.529 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803785' 2026-02-12T18:20:02.529 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.0 seq 25769803785 2026-02-12T18:20:02.529 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:20:02.765 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803785 -lt 25769803785 2026-02-12T18:20:02.765 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:20:02.766 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607561 2026-02-12T18:20:02.766 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:20:02.766 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:20:02.766 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607561 2026-02-12T18:20:02.767 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:20:02.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607561 2026-02-12T18:20:02.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607561' 2026-02-12T18:20:02.768 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.1 seq 51539607561 2026-02-12T18:20:02.769 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:20:02.997 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607559 -lt 51539607561 2026-02-12T18:20:02.997 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:20:03.998 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:20:03.999 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:20:04.228 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607561 -lt 51539607561 2026-02-12T18:20:04.228 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:20:04.229 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-73014444040 2026-02-12T18:20:04.229 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:20:04.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:20:04.230 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444040 2026-02-12T18:20:04.230 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:20:04.231 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=73014444040 2026-02-12T18:20:04.231 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 73014444040' 2026-02-12T18:20:04.232 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.2 seq 73014444040 2026-02-12T18:20:04.232 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:20:04.460 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 73014444040 -lt 73014444040 2026-02-12T18:20:04.460 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:20:04.461 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-103079215112 2026-02-12T18:20:04.461 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:20:04.462 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:20:04.462 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-103079215112 2026-02-12T18:20:04.463 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:20:04.464 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=103079215112 2026-02-12T18:20:04.464 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 103079215112' 2026-02-12T18:20:04.464 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.3 seq 103079215112 2026-02-12T18:20:04.464 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:20:04.696 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 103079215112 -lt 103079215112 2026-02-12T18:20:04.697 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:20:04.697 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-141733920775 2026-02-12T18:20:04.698 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:20:04.698 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-12T18:20:04.699 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-141733920775 2026-02-12T18:20:04.699 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:20:04.700 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=141733920775 2026-02-12T18:20:04.700 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 141733920775' 2026-02-12T18:20:04.700 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.4 seq 141733920775 2026-02-12T18:20:04.700 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-12T18:20:04.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 141733920775 -lt 141733920775 2026-02-12T18:20:04.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:20:04.929 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-176093659144 2026-02-12T18:20:04.929 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:20:04.929 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-12T18:20:04.930 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-176093659144 2026-02-12T18:20:04.930 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:20:04.931 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=176093659144 2026-02-12T18:20:04.931 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 176093659144' 2026-02-12T18:20:04.931 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.5 seq 176093659144 2026-02-12T18:20:04.932 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-12T18:20:05.163 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 176093659144 -lt 176093659144 2026-02-12T18:20:05.163 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:20:05.164 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-210453397510 2026-02-12T18:20:05.164 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:20:05.164 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-12T18:20:05.165 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-210453397510 2026-02-12T18:20:05.165 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:20:05.166 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=210453397510 2026-02-12T18:20:05.166 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 210453397510' 2026-02-12T18:20:05.167 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.6 seq 210453397510 2026-02-12T18:20:05.167 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-12T18:20:05.396 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 210453397510 -lt 210453397510 2026-02-12T18:20:05.397 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:20:05.397 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:20:05.397 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:20:05.712 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:20:05.713 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:20:05.713 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:20:05.713 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:20:05.713 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:20:05.713 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:20:05.713 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:20:05.714 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:20:05.936 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:20:05.937 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:20:05.937 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:20:05.937 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:20:06.253 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:20:06.254 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:20:06.254 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:20:06.254 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:491: TEST_ec_recovery_multiple_objects: rados_get td/test-erasure-eio pool-jerasure test1 2026-02-12T18:20:06.254 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-12T18:20:06.254 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-12T18:20:06.254 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test1 2026-02-12T18:20:06.254 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-12T18:20:06.254 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-12T18:20:06.254 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test1 td/test-erasure-eio/COPY 2026-02-12T18:20:06.270 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-12T18:20:06.271 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-12T18:20:06.272 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:492: TEST_ec_recovery_multiple_objects: rados_get td/test-erasure-eio pool-jerasure test2 2026-02-12T18:20:06.273 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-12T18:20:06.273 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-12T18:20:06.273 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test2 2026-02-12T18:20:06.273 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-12T18:20:06.273 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-12T18:20:06.273 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test2 td/test-erasure-eio/COPY 2026-02-12T18:20:06.288 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-12T18:20:06.289 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-12T18:20:06.290 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:493: TEST_ec_recovery_multiple_objects: rados_get td/test-erasure-eio pool-jerasure test3 2026-02-12T18:20:06.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-12T18:20:06.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-12T18:20:06.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test3 2026-02-12T18:20:06.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-12T18:20:06.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-12T18:20:06.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test3 td/test-erasure-eio/COPY 2026-02-12T18:20:06.307 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-12T18:20:06.307 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-12T18:20:06.308 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:495: TEST_ec_recovery_multiple_objects: delete_erasure_coded_pool pool-jerasure 2026-02-12T18:20:06.309 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-12T18:20:06.309 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-12T18:20:06.581 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-jerasure' does not exist 2026-02-12T18:20:06.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-12T18:20:06.868 INFO:tasks.workunit.client.0.trial116.stderr:erasure-code-profile myprofile does not exist 2026-02-12T18:20:06.877 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-12T18:20:06.878 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-12T18:20:06.878 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:20:06.878 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-12T18:20:06.878 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:20:06.878 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:20:06.879 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:20:06.879 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:20:06.879 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:20:06.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:20:06.989 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:20:06.990 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:20:06.990 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:20:06.992 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:20:06.992 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:20:06.992 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:20:06.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:20:06.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:20:06.994 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:20:06.994 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:20:06.994 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:20:06.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:20:06.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-12T18:20:07.011 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:20:07.011 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:07.012 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:07.012 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76821 2026-02-12T18:20:07.013 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:20:07.013 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:20:07.014 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-12T18:20:07.014 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-12T18:20:07.014 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-12T18:20:07.014 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-12T18:20:07.014 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-12T18:20:07.014 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:20:07.014 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-12T18:20:07.014 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:20:07.014 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:20:07.014 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:20:07.014 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:20:07.014 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:20:07.015 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:20:07.015 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:20:07.016 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:20:07.016 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:20:07.017 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:20:07.017 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:20:07.018 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:20:07.018 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:20:07.018 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:20:07.019 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:20:07.019 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:20:07.019 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:20:07.020 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:20:07.020 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-12T18:20:07.022 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:20:07.022 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:07.022 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:07.022 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76821 2026-02-12T18:20:07.023 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:20:07.023 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:20:07.023 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-12T18:20:07.024 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T18:20:07.024 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:07.024 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:07.024 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76821 2026-02-12T18:20:07.025 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T18:20:07.026 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T18:20:07.026 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T18:20:07.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-12T18:20:07.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-12T18:20:07.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-12T18:20:07.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T18:20:07.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T18:20:07.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T18:20:07.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-12T18:20:07.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-12T18:20:07.045 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T18:20:07.045 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:20:07.045 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:20:07.045 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:20:07.045 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:07.045 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:07.045 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:20:07.046 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T18:20:07.068 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T18:20:07.069 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T18:20:07.069 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:20:07.070 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:20:07.070 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T18:20:07.071 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T18:20:07.071 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:20:07.071 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:20:07.071 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:20:07.071 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:20:07.071 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:07.071 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:07.071 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:20:07.072 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:20:07.072 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76821/ceph-mon.a.asok config get fsid 2026-02-12T18:20:07.115 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T18:20:07.115 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:20:07.115 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:20:07.115 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T18:20:07.115 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T18:20:07.117 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:20:07.117 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:20:07.117 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:20:07.117 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:20:07.117 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:07.117 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:07.117 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:20:07.117 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:20:07.117 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76821/ceph-mon.a.asok config get mon_host 2026-02-12T18:20:07.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-12T18:20:07.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-12T18:20:07.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T18:20:07.159 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T18:20:07.159 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T18:20:07.159 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-12T18:20:07.159 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T18:20:07.259 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T18:20:07.259 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:20:07.259 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:20:07.260 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:20:07.260 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:07.260 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:07.260 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:20:07.261 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:20:07.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:20:07.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-12T18:20:07.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T18:20:07.369 INFO:tasks.workunit.client.0.trial116.stderr:pool 'rbd' already exists 2026-02-12T18:20:07.376 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:20:08.378 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-12T18:20:08.378 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:20:08.379 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:20:08.379 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:20:08.379 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:08.379 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:08.379 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:20:08.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-12T18:20:08.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-mon.a.asok log flush 2026-02-12T18:20:08.418 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:20:08.423 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-12T18:20:08.425 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:20:07.059+0000 7fc4cba3bc00 0 load: jerasure load: lrc 2026-02-12T18:20:08.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_multiple_objects_eio td/test-erasure-eio 2026-02-12T18:20:08.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:500: TEST_ec_recovery_multiple_objects_eio: local dir=td/test-erasure-eio 2026-02-12T18:20:08.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:501: TEST_ec_recovery_multiple_objects_eio: local objname=myobject 2026-02-12T18:20:08.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:503: TEST_ec_recovery_multiple_objects_eio: ORIG_ARGS='--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:20:08.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:504: TEST_ec_recovery_multiple_objects_eio: CEPH_ARGS+=' --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-12T18:20:08.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:505: TEST_ec_recovery_multiple_objects_eio: setup_osds 7 2026-02-12T18:20:08.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-12T18:20:08.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-12T18:20:08.426 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-12T18:20:08.427 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-12T18:20:08.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:20:08.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-12T18:20:08.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:20:08.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:20:08.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T18:20:08.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:20:08.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-12T18:20:08.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-12T18:20:08.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:20:08.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:20:08.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:20:08.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-12T18:20:08.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-12T18:20:08.432 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:20:08.432 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:20:08.432 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:20:08.432 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:20:08.432 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:20:08.432 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:20:08.432 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:20:08.432 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:08.432 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:08.432 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:20:08.432 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:20:08.432 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:20:08.432 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:20:08.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:20:08.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:20:08.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:20:08.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:20:08.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:20:08.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:20:08.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:20:08.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:20:08.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:20:08.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-12T18:20:08.433 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:20:08.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=47ba7b43-ce8a-49cd-8ecb-46f1d748f06d 2026-02-12T18:20:08.434 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 47ba7b43-ce8a-49cd-8ecb-46f1d748f06d' 2026-02-12T18:20:08.434 INFO:tasks.workunit.client.0.trial116.stdout:add osd0 47ba7b43-ce8a-49cd-8ecb-46f1d748f06d 2026-02-12T18:20:08.434 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:20:08.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDYGY5pGSYwGhAAdRNNZbck/oHOrtDMcUWcPw== 2026-02-12T18:20:08.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDYGY5pGSYwGhAAdRNNZbck/oHOrtDMcUWcPw=="}' 2026-02-12T18:20:08.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 47ba7b43-ce8a-49cd-8ecb-46f1d748f06d -i td/test-erasure-eio/0/new.json 2026-02-12T18:20:08.545 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:20:08.552 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-12T18:20:08.553 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDYGY5pGSYwGhAAdRNNZbck/oHOrtDMcUWcPw== --osd-uuid 47ba7b43-ce8a-49cd-8ecb-46f1d748f06d 2026-02-12T18:20:08.571 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:08.569+0000 7f359060c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:08.573 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:08.570+0000 7f359060c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:08.574 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:08.571+0000 7f359060c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:08.574 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:08.571+0000 7f359060c780 -1 bdev(0x4220c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-12T18:20:08.574 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:08.571+0000 7f359060c780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-12T18:20:08.985 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-12T18:20:08.985 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:20:08.985 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T18:20:08.986 INFO:tasks.workunit.client.0.trial116.stdout:adding osd0 key to auth repository 2026-02-12T18:20:08.986 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:20:09.089 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T18:20:09.090 INFO:tasks.workunit.client.0.trial116.stdout:start osd.0 2026-02-12T18:20:09.091 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:20:09.091 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:20:09.091 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:20:09.091 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:20:09.106 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:09.104+0000 7f2f13c62780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:09.108 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:09.105+0000 7f2f13c62780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:09.109 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:09.106+0000 7f2f13c62780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:09.189 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T18:20:09.189 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:20:09.189 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T18:20:09.189 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:20:09.189 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:20:09.190 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:20:09.190 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:20:09.190 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:20:09.190 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:20:09.190 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:20:09.292 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:20:09.314 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:09.312+0000 7f2f13c62780 -1 Falling back to public interface 2026-02-12T18:20:09.576 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:09.573+0000 7f2f13c62780 -1 osd.0 0 log_to_monitors true 2026-02-12T18:20:10.293 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:20:10.293 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:20:10.294 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:20:10.294 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:20:10.295 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:20:10.295 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:20:10.520 INFO:tasks.workunit.client.0.trial116.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3095755017,v1:127.0.0.1:6803/3095755017] [v2:127.0.0.1:6804/3095755017,v1:127.0.0.1:6805/3095755017] exists,up 47ba7b43-ce8a-49cd-8ecb-46f1d748f06d 2026-02-12T18:20:10.525 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:20:10.525 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:20:10.525 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:20:10.525 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:20:10.525 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-12T18:20:10.525 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:20:10.525 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:20:10.525 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-12T18:20:10.525 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:20:10.526 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-12T18:20:10.526 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-12T18:20:10.526 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:20:10.526 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:20:10.526 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:20:10.526 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-12T18:20:10.526 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-12T18:20:10.526 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:20:10.526 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:20:10.526 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:20:10.526 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:20:10.526 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:20:10.526 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:20:10.527 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:20:10.527 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:10.527 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:10.527 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:20:10.527 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:20:10.527 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:20:10.527 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:20:10.527 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:20:10.527 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:20:10.527 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:20:10.527 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:20:10.527 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:20:10.528 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:20:10.528 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:20:10.528 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:20:10.528 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:20:10.528 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-12T18:20:10.528 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:20:10.528 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b753ade4-70af-4b31-ad73-b93e9ea96c9c 2026-02-12T18:20:10.528 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 b753ade4-70af-4b31-ad73-b93e9ea96c9c' 2026-02-12T18:20:10.528 INFO:tasks.workunit.client.0.trial116.stdout:add osd1 b753ade4-70af-4b31-ad73-b93e9ea96c9c 2026-02-12T18:20:10.529 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:20:10.536 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDaGY5pVy7DHxAAwPF1HWl6FuDv1uSmpcGH2A== 2026-02-12T18:20:10.536 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDaGY5pVy7DHxAAwPF1HWl6FuDv1uSmpcGH2A=="}' 2026-02-12T18:20:10.536 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b753ade4-70af-4b31-ad73-b93e9ea96c9c -i td/test-erasure-eio/1/new.json 2026-02-12T18:20:10.772 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:20:10.780 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-12T18:20:10.781 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDaGY5pVy7DHxAAwPF1HWl6FuDv1uSmpcGH2A== --osd-uuid b753ade4-70af-4b31-ad73-b93e9ea96c9c 2026-02-12T18:20:10.799 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:10.796+0000 7f00e7087780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:10.801 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:10.797+0000 7f00e7087780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:10.802 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:10.798+0000 7f00e7087780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:10.802 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:10.798+0000 7f00e7087780 -1 bdev(0x28b6c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-12T18:20:10.802 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:10.798+0000 7f00e7087780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-12T18:20:11.129 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-12T18:20:11.129 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:20:11.130 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-12T18:20:11.130 INFO:tasks.workunit.client.0.trial116.stdout:adding osd1 key to auth repository 2026-02-12T18:20:11.131 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:20:11.453 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-12T18:20:11.454 INFO:tasks.workunit.client.0.trial116.stdout:start osd.1 2026-02-12T18:20:11.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:20:11.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:20:11.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:20:11.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:20:11.470 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:11.466+0000 7fcc3d84b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:11.471 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:11.468+0000 7fcc3d84b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:11.472 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:11.468+0000 7fcc3d84b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:11.614 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:11.610+0000 7fcc3d84b780 -1 Falling back to public interface 2026-02-12T18:20:11.683 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-12T18:20:11.683 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:20:11.683 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T18:20:11.683 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:20:11.683 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:20:11.683 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:20:11.684 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:20:11.684 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:20:11.684 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:20:11.684 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:20:11.911 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:20:11.937 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:11.933+0000 7fcc3d84b780 -1 osd.1 0 log_to_monitors true 2026-02-12T18:20:12.912 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:20:12.913 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:20:12.913 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:20:12.914 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:20:12.914 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:20:12.914 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:20:13.137 INFO:tasks.workunit.client.0.trial116.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2269044835,v1:127.0.0.1:6811/2269044835] [v2:127.0.0.1:6812/2269044835,v1:127.0.0.1:6813/2269044835] exists,up b753ade4-70af-4b31-ad73-b93e9ea96c9c 2026-02-12T18:20:13.141 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:20:13.141 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:20:13.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:20:13.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:20:13.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-12T18:20:13.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:20:13.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:20:13.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-12T18:20:13.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:20:13.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-12T18:20:13.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-12T18:20:13.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:20:13.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:20:13.143 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:20:13.143 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-12T18:20:13.143 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-12T18:20:13.143 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:20:13.143 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:20:13.143 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:20:13.143 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:20:13.143 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:20:13.143 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:20:13.143 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:20:13.143 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:13.143 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:13.143 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:20:13.144 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:20:13.144 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:20:13.144 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:20:13.144 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:20:13.144 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:20:13.144 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:20:13.144 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:20:13.144 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:20:13.144 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:20:13.144 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:20:13.144 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:20:13.144 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:20:13.144 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-12T18:20:13.145 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:20:13.145 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c1e1d713-eb66-41c3-b2d0-4056e44c9616 2026-02-12T18:20:13.145 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 c1e1d713-eb66-41c3-b2d0-4056e44c9616' 2026-02-12T18:20:13.145 INFO:tasks.workunit.client.0.trial116.stdout:add osd2 c1e1d713-eb66-41c3-b2d0-4056e44c9616 2026-02-12T18:20:13.145 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:20:13.152 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDdGY5pEf3mCBAAiZWbCZIM8CxVaFQo0H0DIg== 2026-02-12T18:20:13.152 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDdGY5pEf3mCBAAiZWbCZIM8CxVaFQo0H0DIg=="}' 2026-02-12T18:20:13.152 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c1e1d713-eb66-41c3-b2d0-4056e44c9616 -i td/test-erasure-eio/2/new.json 2026-02-12T18:20:13.383 INFO:tasks.workunit.client.0.trial116.stdout:2 2026-02-12T18:20:13.391 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-12T18:20:13.392 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDdGY5pEf3mCBAAiZWbCZIM8CxVaFQo0H0DIg== --osd-uuid c1e1d713-eb66-41c3-b2d0-4056e44c9616 2026-02-12T18:20:13.410 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:13.407+0000 7f781563e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:13.412 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:13.408+0000 7f781563e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:13.413 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:13.409+0000 7f781563e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:13.413 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:13.409+0000 7f781563e780 -1 bdev(0x3af8c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-12T18:20:13.413 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:13.409+0000 7f781563e780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-12T18:20:13.728 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-12T18:20:13.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:20:13.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-12T18:20:13.729 INFO:tasks.workunit.client.0.trial116.stdout:adding osd2 key to auth repository 2026-02-12T18:20:13.730 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:20:14.058 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-12T18:20:14.059 INFO:tasks.workunit.client.0.trial116.stdout:start osd.2 2026-02-12T18:20:14.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:20:14.060 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:20:14.060 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:20:14.060 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:20:14.075 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:14.071+0000 7fd2b9a1a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:14.076 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:14.073+0000 7fd2b9a1a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:14.077 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:14.074+0000 7fd2b9a1a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:14.219 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:14.216+0000 7fd2b9a1a780 -1 Falling back to public interface 2026-02-12T18:20:14.285 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-12T18:20:14.285 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:20:14.285 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T18:20:14.285 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:20:14.285 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:20:14.285 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:20:14.285 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:20:14.286 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:20:14.286 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:20:14.286 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:20:14.514 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:20:14.547 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:14.543+0000 7fd2b9a1a780 -1 osd.2 0 log_to_monitors true 2026-02-12T18:20:15.515 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:20:15.516 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:20:15.516 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:20:15.517 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:20:15.517 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:20:15.517 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:20:15.742 INFO:tasks.workunit.client.0.trial116.stdout:osd.2 up in weight 1 up_from 19 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2893813708,v1:127.0.0.1:6819/2893813708] [v2:127.0.0.1:6820/2893813708,v1:127.0.0.1:6821/2893813708] exists,up c1e1d713-eb66-41c3-b2d0-4056e44c9616 2026-02-12T18:20:15.746 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:20:15.746 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:20:15.746 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:20:15.746 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:20:15.746 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-12T18:20:15.746 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:20:15.746 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:20:15.746 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-12T18:20:15.746 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:20:15.746 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-12T18:20:15.747 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-12T18:20:15.747 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:20:15.747 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:20:15.747 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:20:15.747 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-12T18:20:15.747 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-12T18:20:15.747 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:20:15.747 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:20:15.747 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:20:15.747 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:20:15.747 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:20:15.747 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:20:15.748 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:20:15.748 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:15.748 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:15.748 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:20:15.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:20:15.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:20:15.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:20:15.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:20:15.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:20:15.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:20:15.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:20:15.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:20:15.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:20:15.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:20:15.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:20:15.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:20:15.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-12T18:20:15.749 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:20:15.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=63b2ab11-a970-40be-9f1c-c96afee7b4f0 2026-02-12T18:20:15.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 63b2ab11-a970-40be-9f1c-c96afee7b4f0' 2026-02-12T18:20:15.749 INFO:tasks.workunit.client.0.trial116.stdout:add osd3 63b2ab11-a970-40be-9f1c-c96afee7b4f0 2026-02-12T18:20:15.750 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:20:15.757 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDfGY5pIqPyLBAAKR0fyS9aQw5ONnn/3hoX6w== 2026-02-12T18:20:15.757 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDfGY5pIqPyLBAAKR0fyS9aQw5ONnn/3hoX6w=="}' 2026-02-12T18:20:15.757 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 63b2ab11-a970-40be-9f1c-c96afee7b4f0 -i td/test-erasure-eio/3/new.json 2026-02-12T18:20:16.000 INFO:tasks.workunit.client.0.trial116.stdout:3 2026-02-12T18:20:16.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-12T18:20:16.008 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDfGY5pIqPyLBAAKR0fyS9aQw5ONnn/3hoX6w== --osd-uuid 63b2ab11-a970-40be-9f1c-c96afee7b4f0 2026-02-12T18:20:16.027 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:16.023+0000 7fd62082c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:16.028 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:16.025+0000 7fd62082c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:16.029 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:16.026+0000 7fd62082c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:16.030 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:16.026+0000 7fd62082c780 -1 bdev(0x419ac00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-12T18:20:16.030 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:16.026+0000 7fd62082c780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-12T18:20:16.360 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-12T18:20:16.361 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:20:16.361 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-12T18:20:16.362 INFO:tasks.workunit.client.0.trial116.stdout:adding osd3 key to auth repository 2026-02-12T18:20:16.362 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:20:16.687 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-12T18:20:16.687 INFO:tasks.workunit.client.0.trial116.stdout:start osd.3 2026-02-12T18:20:16.688 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:20:16.688 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:20:16.688 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:20:16.689 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:20:16.705 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:16.701+0000 7f6bfe83b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:16.706 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:16.702+0000 7f6bfe83b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:16.707 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:16.703+0000 7f6bfe83b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:16.840 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:16.836+0000 7f6bfe83b780 -1 Falling back to public interface 2026-02-12T18:20:16.921 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-12T18:20:16.921 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:20:16.921 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-12T18:20:16.921 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:20:16.921 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:20:16.921 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:20:16.921 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:20:16.922 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:20:16.922 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:20:16.922 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:20:17.149 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:20:17.150 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:17.146+0000 7f6bfe83b780 -1 osd.3 0 log_to_monitors true 2026-02-12T18:20:18.150 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:20:18.150 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:20:18.150 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:20:18.150 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:20:18.150 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:20:18.150 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:20:18.384 INFO:tasks.workunit.client.0.trial116.stdout:osd.3 up in weight 1 up_from 27 up_thru 28 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2741869028,v1:127.0.0.1:6827/2741869028] [v2:127.0.0.1:6828/2741869028,v1:127.0.0.1:6829/2741869028] exists,up 63b2ab11-a970-40be-9f1c-c96afee7b4f0 2026-02-12T18:20:18.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:20:18.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:20:18.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:20:18.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:20:18.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-12T18:20:18.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:20:18.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:20:18.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-12T18:20:18.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:20:18.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-12T18:20:18.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-12T18:20:18.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:20:18.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:20:18.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:20:18.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-12T18:20:18.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-12T18:20:18.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:20:18.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:20:18.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:20:18.386 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:20:18.386 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:20:18.386 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:20:18.386 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:20:18.386 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:18.386 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:18.386 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:20:18.386 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:20:18.386 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:20:18.386 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:20:18.386 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:20:18.386 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:20:18.386 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:20:18.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:20:18.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:20:18.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:20:18.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:20:18.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:20:18.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:20:18.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-12T18:20:18.388 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:20:18.388 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=71099d17-a99d-4875-b17b-438700d94a6a 2026-02-12T18:20:18.388 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 71099d17-a99d-4875-b17b-438700d94a6a' 2026-02-12T18:20:18.388 INFO:tasks.workunit.client.0.trial116.stdout:add osd4 71099d17-a99d-4875-b17b-438700d94a6a 2026-02-12T18:20:18.389 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:20:18.400 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDiGY5p4MC0FxAAl9ydIuK3Hmng4PLt/UCjiA== 2026-02-12T18:20:18.400 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDiGY5p4MC0FxAAl9ydIuK3Hmng4PLt/UCjiA=="}' 2026-02-12T18:20:18.400 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 71099d17-a99d-4875-b17b-438700d94a6a -i td/test-erasure-eio/4/new.json 2026-02-12T18:20:18.632 INFO:tasks.workunit.client.0.trial116.stdout:4 2026-02-12T18:20:18.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-12T18:20:18.641 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDiGY5p4MC0FxAAl9ydIuK3Hmng4PLt/UCjiA== --osd-uuid 71099d17-a99d-4875-b17b-438700d94a6a 2026-02-12T18:20:18.660 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:18.656+0000 7fa2a8ffd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:18.661 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:18.657+0000 7fa2a8ffd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:18.662 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:18.658+0000 7fa2a8ffd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:18.662 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:18.659+0000 7fa2a8ffd780 -1 bdev(0x3508c00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-12T18:20:18.662 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:18.659+0000 7fa2a8ffd780 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-12T18:20:18.990 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-12T18:20:18.990 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:20:18.991 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-12T18:20:18.991 INFO:tasks.workunit.client.0.trial116.stdout:adding osd4 key to auth repository 2026-02-12T18:20:18.992 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:20:19.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-12T18:20:19.314 INFO:tasks.workunit.client.0.trial116.stdout:start osd.4 2026-02-12T18:20:19.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:20:19.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:20:19.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:20:19.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:20:19.332 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:19.328+0000 7f079102c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:19.333 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:19.330+0000 7f079102c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:19.334 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:19.331+0000 7f079102c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:19.463 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:19.460+0000 7f079102c780 -1 Falling back to public interface 2026-02-12T18:20:19.544 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-12T18:20:19.544 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:20:19.544 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-12T18:20:19.544 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:20:19.544 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:20:19.544 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:20:19.544 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:20:19.544 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:20:19.544 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:20:19.544 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-12T18:20:19.767 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:19.764+0000 7f079102c780 -1 osd.4 0 log_to_monitors true 2026-02-12T18:20:19.774 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:20:20.775 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:20:20.776 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:20:20.777 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:20:20.777 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:20:20.777 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:20:20.777 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-12T18:20:21.002 INFO:tasks.workunit.client.0.trial116.stdout:osd.4 up in weight 1 up_from 35 up_thru 36 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/2968923216,v1:127.0.0.1:6835/2968923216] [v2:127.0.0.1:6836/2968923216,v1:127.0.0.1:6837/2968923216] exists,up 71099d17-a99d-4875-b17b-438700d94a6a 2026-02-12T18:20:21.006 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:20:21.006 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:20:21.006 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:20:21.006 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:20:21.006 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-12T18:20:21.006 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:20:21.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:20:21.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-12T18:20:21.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:20:21.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-12T18:20:21.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-12T18:20:21.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:20:21.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:20:21.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:20:21.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-12T18:20:21.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-12T18:20:21.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:20:21.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:20:21.008 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:20:21.008 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:20:21.008 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:20:21.008 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:20:21.008 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:20:21.008 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:21.008 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:21.008 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:20:21.008 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:20:21.008 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:20:21.008 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:20:21.008 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:20:21.008 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:20:21.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:20:21.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:20:21.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:20:21.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:20:21.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:20:21.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:20:21.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:20:21.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-12T18:20:21.009 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:20:21.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0276f590-4ab5-409b-98ea-40e80dd179cc 2026-02-12T18:20:21.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 0276f590-4ab5-409b-98ea-40e80dd179cc' 2026-02-12T18:20:21.010 INFO:tasks.workunit.client.0.trial116.stdout:add osd5 0276f590-4ab5-409b-98ea-40e80dd179cc 2026-02-12T18:20:21.010 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:20:21.017 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDlGY5p3MHXABAAjTlcLzGwWKRW94c0Jth2bg== 2026-02-12T18:20:21.017 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDlGY5p3MHXABAAjTlcLzGwWKRW94c0Jth2bg=="}' 2026-02-12T18:20:21.017 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0276f590-4ab5-409b-98ea-40e80dd179cc -i td/test-erasure-eio/5/new.json 2026-02-12T18:20:21.260 INFO:tasks.workunit.client.0.trial116.stdout:5 2026-02-12T18:20:21.269 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-12T18:20:21.270 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDlGY5p3MHXABAAjTlcLzGwWKRW94c0Jth2bg== --osd-uuid 0276f590-4ab5-409b-98ea-40e80dd179cc 2026-02-12T18:20:21.288 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:21.285+0000 7fd29ee44780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:21.290 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:21.286+0000 7fd29ee44780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:21.291 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:21.287+0000 7fd29ee44780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:21.291 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:21.287+0000 7fd29ee44780 -1 bdev(0x423ac00 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-12T18:20:21.291 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:21.287+0000 7fd29ee44780 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-12T18:20:21.607 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-12T18:20:21.607 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:20:21.608 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-12T18:20:21.608 INFO:tasks.workunit.client.0.trial116.stdout:adding osd5 key to auth repository 2026-02-12T18:20:21.609 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:20:21.932 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-12T18:20:21.932 INFO:tasks.workunit.client.0.trial116.stdout:start osd.5 2026-02-12T18:20:21.933 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:20:21.933 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:20:21.933 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:20:21.933 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:20:21.950 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:21.946+0000 7f1c32a17780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:21.951 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:21.947+0000 7f1c32a17780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:21.952 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:21.948+0000 7f1c32a17780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:22.080 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:22.077+0000 7f1c32a17780 -1 Falling back to public interface 2026-02-12T18:20:22.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-12T18:20:22.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:20:22.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-12T18:20:22.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:20:22.161 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:20:22.161 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:20:22.161 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:20:22.161 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:20:22.161 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:20:22.162 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-12T18:20:22.395 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:20:22.407 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:22.404+0000 7f1c32a17780 -1 osd.5 0 log_to_monitors true 2026-02-12T18:20:23.396 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:20:23.396 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:20:23.396 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:20:23.396 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:20:23.396 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:20:23.396 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-12T18:20:23.621 INFO:tasks.workunit.client.0.trial116.stdout:osd.5 up in weight 1 up_from 44 up_thru 45 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/1169016677,v1:127.0.0.1:6843/1169016677] [v2:127.0.0.1:6844/1169016677,v1:127.0.0.1:6845/1169016677] exists,up 0276f590-4ab5-409b-98ea-40e80dd179cc 2026-02-12T18:20:23.624 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:20:23.625 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:20:23.625 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:20:23.625 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:20:23.625 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-12T18:20:23.625 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:20:23.625 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:20:23.625 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-12T18:20:23.625 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:20:23.625 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-12T18:20:23.625 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-12T18:20:23.626 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:20:23.626 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:20:23.626 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:20:23.626 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-12T18:20:23.626 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-12T18:20:23.626 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:20:23.626 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:20:23.626 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:20:23.626 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:20:23.626 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:20:23.626 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:20:23.626 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:20:23.627 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:23.627 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:23.627 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:20:23.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:20:23.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:20:23.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:20:23.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:20:23.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:20:23.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:20:23.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:20:23.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:20:23.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:20:23.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:20:23.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:20:23.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:20:23.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-12T18:20:23.628 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:20:23.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7b5f0577-41df-4151-aa31-fa7b03bfaebd 2026-02-12T18:20:23.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 7b5f0577-41df-4151-aa31-fa7b03bfaebd' 2026-02-12T18:20:23.629 INFO:tasks.workunit.client.0.trial116.stdout:add osd6 7b5f0577-41df-4151-aa31-fa7b03bfaebd 2026-02-12T18:20:23.629 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:20:23.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDnGY5prZC6JRAAEtFB/HneDZNjug7dLnpmRg== 2026-02-12T18:20:23.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDnGY5prZC6JRAAEtFB/HneDZNjug7dLnpmRg=="}' 2026-02-12T18:20:23.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7b5f0577-41df-4151-aa31-fa7b03bfaebd -i td/test-erasure-eio/6/new.json 2026-02-12T18:20:23.870 INFO:tasks.workunit.client.0.trial116.stdout:6 2026-02-12T18:20:23.878 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-12T18:20:23.879 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDnGY5prZC6JRAAEtFB/HneDZNjug7dLnpmRg== --osd-uuid 7b5f0577-41df-4151-aa31-fa7b03bfaebd 2026-02-12T18:20:23.897 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:23.894+0000 7ff13ea2f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:23.898 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:23.895+0000 7ff13ea2f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:23.899 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:23.896+0000 7ff13ea2f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:23.900 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:23.896+0000 7ff13ea2f780 -1 bdev(0x326cc00 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-12T18:20:23.900 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:23.896+0000 7ff13ea2f780 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-12T18:20:24.209 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-12T18:20:24.209 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:20:24.210 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-12T18:20:24.210 INFO:tasks.workunit.client.0.trial116.stdout:adding osd6 key to auth repository 2026-02-12T18:20:24.210 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:20:24.530 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-12T18:20:24.531 INFO:tasks.workunit.client.0.trial116.stdout:start osd.6 2026-02-12T18:20:24.531 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:20:24.531 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:20:24.532 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:20:24.532 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:20:24.548 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:24.544+0000 7f038062c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:24.549 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:24.546+0000 7f038062c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:24.550 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:24.546+0000 7f038062c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:24.683 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:24.679+0000 7f038062c780 -1 Falling back to public interface 2026-02-12T18:20:24.759 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-12T18:20:24.759 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:20:24.759 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-12T18:20:24.760 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:20:24.760 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:20:24.760 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:20:24.760 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:20:24.761 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:20:24.761 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:20:24.761 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-12T18:20:24.985 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:20:24.995 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:24.991+0000 7f038062c780 -1 osd.6 0 log_to_monitors true 2026-02-12T18:20:25.987 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:20:25.987 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:20:25.987 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:20:25.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:20:25.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:20:25.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-12T18:20:26.212 INFO:tasks.workunit.client.0.trial116.stdout:osd.6 up in weight 1 up_from 52 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/1274907190,v1:127.0.0.1:6851/1274907190] [v2:127.0.0.1:6852/1274907190,v1:127.0.0.1:6853/1274907190] exists,up 7b5f0577-41df-4151-aa31-fa7b03bfaebd 2026-02-12T18:20:26.214 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:20:26.214 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:20:26.214 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:20:26.214 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-12T18:20:26.214 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T18:20:26.214 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T18:20:26.215 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:20:26.215 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:26.215 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:26.215 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.0.asok 2026-02-12T18:20:26.216 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-12T18:20:26.216 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-osd.0.asok log flush 2026-02-12T18:20:26.258 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:20:26.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-12T18:20:26.265 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:20:09.314+0000 7f2f13c62780 0 load: jerasure load: lrc 2026-02-12T18:20:26.266 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:506: TEST_ec_recovery_multiple_objects_eio: CEPH_ARGS='--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:20:26.266 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:508: TEST_ec_recovery_multiple_objects_eio: local poolname=pool-jerasure 2026-02-12T18:20:26.266 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:509: TEST_ec_recovery_multiple_objects_eio: create_erasure_coded_pool pool-jerasure 3 2 2026-02-12T18:20:26.266 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-12T18:20:26.266 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-12T18:20:26.266 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-12T18:20:26.266 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-12T18:20:26.266 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-12T18:20:26.267 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-12T18:20:26.267 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-12T18:20:26.551 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-12T18:20:26.551 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-12T18:20:26.922 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-jerasure' already exists 2026-02-12T18:20:26.929 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:20:27.931 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-12T18:20:27.931 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:20:27.931 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:20:27.932 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:20:27.932 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:20:27.933 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:20:27.933 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:20:27.933 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:20:27.933 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:20:27.933 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:20:28.018 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:20:28.018 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:20:28.019 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:20:28.019 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:20:28.019 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:20:28.019 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:20:28.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:20:28.244 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:20:28.244 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:20:28.244 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:20:28.245 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:20:28.245 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:20:28.245 INFO:tasks.workunit.client.0.trial116.stderr:6' 2026-02-12T18:20:28.245 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:20:28.245 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:20:28.245 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:20:28.309 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803782 2026-02-12T18:20:28.309 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803782 2026-02-12T18:20:28.310 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782' 2026-02-12T18:20:28.310 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:20:28.310 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:20:28.374 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574854 2026-02-12T18:20:28.375 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574854 2026-02-12T18:20:28.375 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574854' 2026-02-12T18:20:28.375 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:20:28.375 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:20:28.436 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378629 2026-02-12T18:20:28.437 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378629 2026-02-12T18:20:28.437 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-81604378629' 2026-02-12T18:20:28.437 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:20:28.437 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:20:28.504 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116997 2026-02-12T18:20:28.504 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116997 2026-02-12T18:20:28.505 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-81604378629 3-115964116997' 2026-02-12T18:20:28.505 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:20:28.505 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-12T18:20:28.569 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855365 2026-02-12T18:20:28.569 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855365 2026-02-12T18:20:28.569 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-81604378629 3-115964116997 4-150323855365' 2026-02-12T18:20:28.569 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:20:28.569 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-12T18:20:28.632 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=188978561029 2026-02-12T18:20:28.632 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 188978561029 2026-02-12T18:20:28.633 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-81604378629 3-115964116997 4-150323855365 5-188978561029' 2026-02-12T18:20:28.633 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:20:28.633 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-12T18:20:28.697 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299396 2026-02-12T18:20:28.698 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299396 2026-02-12T18:20:28.698 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-81604378629 3-115964116997 4-150323855365 5-188978561029 6-223338299396' 2026-02-12T18:20:28.698 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:20:28.698 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803782 2026-02-12T18:20:28.698 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:20:28.700 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:20:28.700 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803782 2026-02-12T18:20:28.700 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:20:28.700 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803782 2026-02-12T18:20:28.701 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.0 seq 25769803782 2026-02-12T18:20:28.701 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803782' 2026-02-12T18:20:28.702 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:20:28.929 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803780 -lt 25769803782 2026-02-12T18:20:28.929 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:20:29.930 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:20:29.930 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:20:30.156 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803782 2026-02-12T18:20:30.156 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:20:30.157 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574854 2026-02-12T18:20:30.157 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:20:30.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:20:30.159 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574854 2026-02-12T18:20:30.159 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:20:30.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574854 2026-02-12T18:20:30.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574854' 2026-02-12T18:20:30.160 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.1 seq 55834574854 2026-02-12T18:20:30.161 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:20:30.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574854 -lt 55834574854 2026-02-12T18:20:30.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:20:30.388 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-81604378629 2026-02-12T18:20:30.388 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:20:30.389 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:20:30.389 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-81604378629 2026-02-12T18:20:30.389 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:20:30.391 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=81604378629 2026-02-12T18:20:30.391 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 81604378629' 2026-02-12T18:20:30.391 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.2 seq 81604378629 2026-02-12T18:20:30.391 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:20:30.616 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 81604378629 -lt 81604378629 2026-02-12T18:20:30.617 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:20:30.617 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-115964116997 2026-02-12T18:20:30.617 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:20:30.618 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:20:30.618 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964116997 2026-02-12T18:20:30.618 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:20:30.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964116997 2026-02-12T18:20:30.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 115964116997' 2026-02-12T18:20:30.620 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.3 seq 115964116997 2026-02-12T18:20:30.620 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:20:30.845 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964116997 -lt 115964116997 2026-02-12T18:20:30.845 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:20:30.846 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-150323855365 2026-02-12T18:20:30.846 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:20:30.846 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-12T18:20:30.847 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-150323855365 2026-02-12T18:20:30.847 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:20:30.848 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=150323855365 2026-02-12T18:20:30.848 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 150323855365' 2026-02-12T18:20:30.848 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.4 seq 150323855365 2026-02-12T18:20:30.849 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-12T18:20:31.077 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 150323855365 -lt 150323855365 2026-02-12T18:20:31.077 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:20:31.077 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-188978561029 2026-02-12T18:20:31.078 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:20:31.078 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-12T18:20:31.079 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-188978561029 2026-02-12T18:20:31.079 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:20:31.080 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=188978561029 2026-02-12T18:20:31.080 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 188978561029' 2026-02-12T18:20:31.080 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.5 seq 188978561029 2026-02-12T18:20:31.081 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-12T18:20:31.305 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 188978561029 -lt 188978561029 2026-02-12T18:20:31.305 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:20:31.306 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-223338299396 2026-02-12T18:20:31.306 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:20:31.306 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-12T18:20:31.307 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-223338299396 2026-02-12T18:20:31.307 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:20:31.308 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=223338299396 2026-02-12T18:20:31.308 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 223338299396' 2026-02-12T18:20:31.309 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.6 seq 223338299396 2026-02-12T18:20:31.309 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-12T18:20:31.536 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 223338299397 -lt 223338299396 2026-02-12T18:20:31.536 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:20:31.537 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:20:31.537 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:20:31.854 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:20:31.855 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:20:31.855 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:20:31.855 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:20:31.855 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:20:31.855 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:20:31.855 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:20:31.855 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:20:32.079 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:20:32.079 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:20:32.080 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:20:32.080 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:20:32.400 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:20:32.400 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:20:32.400 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:20:32.400 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:511: TEST_ec_recovery_multiple_objects_eio: rados_put td/test-erasure-eio pool-jerasure test1 2026-02-12T18:20:32.400 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-12T18:20:32.401 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-12T18:20:32.401 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test1 2026-02-12T18:20:32.401 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:20:32.401 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-12T18:20:32.401 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:20:32.401 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-12T18:20:32.401 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:20:32.401 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-12T18:20:32.401 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:20:32.401 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-12T18:20:32.401 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test1 td/test-erasure-eio/ORIGINAL 2026-02-12T18:20:32.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:512: TEST_ec_recovery_multiple_objects_eio: rados_put td/test-erasure-eio pool-jerasure test2 2026-02-12T18:20:32.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-12T18:20:32.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-12T18:20:32.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test2 2026-02-12T18:20:32.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:20:32.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-12T18:20:32.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:20:32.420 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-12T18:20:32.420 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:20:32.420 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-12T18:20:32.420 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:20:32.420 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-12T18:20:32.420 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test2 td/test-erasure-eio/ORIGINAL 2026-02-12T18:20:32.437 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:513: TEST_ec_recovery_multiple_objects_eio: rados_put td/test-erasure-eio pool-jerasure test3 2026-02-12T18:20:32.437 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-12T18:20:32.437 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-12T18:20:32.437 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test3 2026-02-12T18:20:32.437 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:20:32.437 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-12T18:20:32.437 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:20:32.437 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-12T18:20:32.438 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:20:32.438 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-12T18:20:32.438 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:20:32.438 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-12T18:20:32.438 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test3 td/test-erasure-eio/ORIGINAL 2026-02-12T18:20:32.454 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:516: TEST_ec_recovery_multiple_objects_eio: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 0 2026-02-12T18:20:32.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-12T18:20:32.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-12T18:20:32.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-12T18:20:32.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-12T18:20:32.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-12T18:20:32.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-12T18:20:32.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=myobject 2026-02-12T18:20:32.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-12T18:20:32.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-12T18:20:32.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-12T18:20:32.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=0 2026-02-12T18:20:32.456 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-12T18:20:32.456 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure myobject 2026-02-12T18:20:32.456 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:20:32.456 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-12T18:20:32.456 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-12T18:20:32.456 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:20:32.683 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:20:32.683 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:20:32.683 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:20:32.684 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:20:32.684 INFO:tasks.workunit.client.0.trial116.stderr:6' 2026-02-12T18:20:32.684 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2026-02-12T18:20:32.684 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '5' '1' '0' '6') 2026-02-12T18:20:32.684 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-12T18:20:32.684 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=3 2026-02-12T18:20:32.684 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-12T18:20:32.684 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/3/type 2026-02-12T18:20:32.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-12T18:20:32.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-12T18:20:32.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-12T18:20:32.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-12T18:20:32.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-12T18:20:32.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-12T18:20:32.686 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-12T18:20:32.687 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-12T18:20:32.687 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-12T18:20:32.687 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-12T18:20:32.687 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:20:32.687 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:32.687 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:32.687 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.3.asok 2026-02-12T18:20:32.687 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.76821/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-12T18:20:32.732 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-12T18:20:32.733 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-12T18:20:32.734 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-12T18:20:32.734 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.3 2026-02-12T18:20:32.734 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-12T18:20:32.734 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-12T18:20:32.734 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:20:32.734 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:32.734 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:32.734 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.3.asok 2026-02-12T18:20:32.735 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-12T18:20:32.735 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-osd.3.asok injectdataerr pool-jerasure myobject 0 2026-02-12T18:20:32.781 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:517: TEST_ec_recovery_multiple_objects_eio: ceph osd out 0 2026-02-12T18:20:33.059 INFO:tasks.workunit.client.0.trial116.stderr:osd.0 is already out. 2026-02-12T18:20:33.068 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:520: TEST_ec_recovery_multiple_objects_eio: wait_for_clean 2026-02-12T18:20:33.068 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:20:33.068 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:20:33.068 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:20:33.068 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:20:33.069 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:20:33.069 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:20:33.069 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:20:33.070 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:20:33.070 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:20:33.155 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:20:33.155 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:20:33.155 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:20:33.155 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:20:33.155 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:20:33.156 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:20:33.382 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:20:33.382 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:20:33.382 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:20:33.382 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:20:33.382 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:20:33.383 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:20:33.383 INFO:tasks.workunit.client.0.trial116.stderr:6' 2026-02-12T18:20:33.383 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:20:33.383 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:20:33.383 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:20:33.447 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803785 2026-02-12T18:20:33.447 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803785 2026-02-12T18:20:33.447 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785' 2026-02-12T18:20:33.448 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:20:33.448 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:20:33.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574857 2026-02-12T18:20:33.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574857 2026-02-12T18:20:33.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574857' 2026-02-12T18:20:33.512 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:20:33.512 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:20:33.579 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378632 2026-02-12T18:20:33.579 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378632 2026-02-12T18:20:33.579 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-81604378632' 2026-02-12T18:20:33.579 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:20:33.579 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:20:33.643 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964117000 2026-02-12T18:20:33.643 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964117000 2026-02-12T18:20:33.643 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-81604378632 3-115964117000' 2026-02-12T18:20:33.643 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:20:33.643 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-12T18:20:33.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855368 2026-02-12T18:20:33.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855368 2026-02-12T18:20:33.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-81604378632 3-115964117000 4-150323855368' 2026-02-12T18:20:33.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:20:33.708 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-12T18:20:33.775 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=188978561032 2026-02-12T18:20:33.775 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 188978561032 2026-02-12T18:20:33.775 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-81604378632 3-115964117000 4-150323855368 5-188978561032' 2026-02-12T18:20:33.775 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:20:33.775 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-12T18:20:33.838 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299399 2026-02-12T18:20:33.839 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299399 2026-02-12T18:20:33.839 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-81604378632 3-115964117000 4-150323855368 5-188978561032 6-223338299399' 2026-02-12T18:20:33.839 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:20:33.839 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803785 2026-02-12T18:20:33.839 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:20:33.840 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:20:33.841 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803785 2026-02-12T18:20:33.841 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:20:33.842 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803785 2026-02-12T18:20:33.842 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803785' 2026-02-12T18:20:33.842 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.0 seq 25769803785 2026-02-12T18:20:33.842 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:20:34.070 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 0 -lt 25769803785 2026-02-12T18:20:34.070 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:20:35.071 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:20:35.072 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:20:35.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 0 -lt 25769803785 2026-02-12T18:20:35.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:20:36.300 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-12T18:20:36.301 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:20:36.526 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803785 -lt 25769803785 2026-02-12T18:20:36.526 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:20:36.527 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574857 2026-02-12T18:20:36.527 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:20:36.527 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:20:36.528 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574857 2026-02-12T18:20:36.528 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:20:36.529 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574857 2026-02-12T18:20:36.529 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574857' 2026-02-12T18:20:36.530 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.1 seq 55834574857 2026-02-12T18:20:36.530 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:20:36.755 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574857 -lt 55834574857 2026-02-12T18:20:36.755 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:20:36.756 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-81604378632 2026-02-12T18:20:36.756 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:20:36.756 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:20:36.757 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-81604378632 2026-02-12T18:20:36.757 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:20:36.758 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=81604378632 2026-02-12T18:20:36.758 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 81604378632' 2026-02-12T18:20:36.759 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.2 seq 81604378632 2026-02-12T18:20:36.759 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:20:36.982 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 81604378632 -lt 81604378632 2026-02-12T18:20:36.983 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:20:36.983 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-115964117000 2026-02-12T18:20:36.983 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:20:36.984 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:20:36.984 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964117000 2026-02-12T18:20:36.984 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:20:36.986 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964117000 2026-02-12T18:20:36.986 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.3 seq 115964117000 2026-02-12T18:20:36.986 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 115964117000' 2026-02-12T18:20:36.987 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:20:37.214 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964117000 -lt 115964117000 2026-02-12T18:20:37.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:20:37.215 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-150323855368 2026-02-12T18:20:37.215 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:20:37.216 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-12T18:20:37.216 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-150323855368 2026-02-12T18:20:37.217 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:20:37.218 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=150323855368 2026-02-12T18:20:37.218 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 150323855368' 2026-02-12T18:20:37.218 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.4 seq 150323855368 2026-02-12T18:20:37.218 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-12T18:20:37.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 150323855369 -lt 150323855368 2026-02-12T18:20:37.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:20:37.444 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-188978561032 2026-02-12T18:20:37.444 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:20:37.445 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-12T18:20:37.445 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-188978561032 2026-02-12T18:20:37.445 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:20:37.446 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=188978561032 2026-02-12T18:20:37.447 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 188978561032' 2026-02-12T18:20:37.447 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.5 seq 188978561032 2026-02-12T18:20:37.447 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-12T18:20:37.673 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 188978561032 -lt 188978561032 2026-02-12T18:20:37.673 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:20:37.674 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-223338299399 2026-02-12T18:20:37.674 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:20:37.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-12T18:20:37.675 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-223338299399 2026-02-12T18:20:37.675 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:20:37.676 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=223338299399 2026-02-12T18:20:37.676 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 223338299399' 2026-02-12T18:20:37.677 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.6 seq 223338299399 2026-02-12T18:20:37.677 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-12T18:20:37.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 223338299400 -lt 223338299399 2026-02-12T18:20:37.904 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:20:37.904 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:20:37.904 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:20:38.216 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:20:38.216 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:20:38.217 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:20:38.217 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:20:38.217 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:20:38.217 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:20:38.217 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:20:38.217 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:20:38.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-12T18:20:38.442 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:20:38.443 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:20:38.443 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:20:38.759 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-12T18:20:38.759 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' -1 2026-02-12T18:20:38.759 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-12T18:20:38.760 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=4 2026-02-12T18:20:38.760 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-12T18:20:38.760 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-12T18:20:38.860 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-12T18:20:38.861 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:20:38.861 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:20:38.861 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:20:38.861 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:20:38.861 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:20:38.861 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:20:38.861 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:20:39.086 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-12T18:20:39.087 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:20:39.087 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:20:39.088 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:20:39.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-12T18:20:39.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-12T18:20:39.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-12T18:20:39.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-12T18:20:39.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-12T18:20:39.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-12T18:20:39.410 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-12T18:20:39.410 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-12T18:20:39.410 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-12T18:20:39.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-12T18:20:39.732 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-12T18:20:39.732 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 13 )) 2026-02-12T18:20:39.732 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-12T18:20:39.732 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-12T18:20:39.933 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-12T18:20:39.933 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:20:39.933 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:20:39.933 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:20:39.933 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:20:39.933 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:20:39.934 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:20:39.934 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:20:40.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:20:40.158 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:20:40.159 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:20:40.159 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:20:40.475 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:20:40.475 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:20:40.475 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:20:40.475 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:522: TEST_ec_recovery_multiple_objects_eio: rados_get td/test-erasure-eio pool-jerasure test1 2026-02-12T18:20:40.475 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-12T18:20:40.476 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-12T18:20:40.476 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test1 2026-02-12T18:20:40.476 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-12T18:20:40.476 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-12T18:20:40.476 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test1 td/test-erasure-eio/COPY 2026-02-12T18:20:40.492 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-12T18:20:40.507 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-12T18:20:40.507 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:523: TEST_ec_recovery_multiple_objects_eio: rados_get td/test-erasure-eio pool-jerasure test2 2026-02-12T18:20:40.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-12T18:20:40.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-12T18:20:40.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test2 2026-02-12T18:20:40.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-12T18:20:40.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-12T18:20:40.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test2 td/test-erasure-eio/COPY 2026-02-12T18:20:40.510 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-12T18:20:40.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-12T18:20:40.513 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:524: TEST_ec_recovery_multiple_objects_eio: rados_get td/test-erasure-eio pool-jerasure test3 2026-02-12T18:20:40.513 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-12T18:20:40.513 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-12T18:20:40.514 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test3 2026-02-12T18:20:40.514 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-12T18:20:40.514 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-12T18:20:40.514 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test3 td/test-erasure-eio/COPY 2026-02-12T18:20:40.529 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-12T18:20:40.530 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-12T18:20:40.531 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:526: TEST_ec_recovery_multiple_objects_eio: delete_erasure_coded_pool pool-jerasure 2026-02-12T18:20:40.531 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-12T18:20:40.531 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-12T18:20:40.806 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-jerasure' does not exist 2026-02-12T18:20:40.815 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-12T18:20:41.093 INFO:tasks.workunit.client.0.trial116.stderr:erasure-code-profile myprofile does not exist 2026-02-12T18:20:41.102 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-12T18:20:41.102 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-12T18:20:41.103 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:20:41.103 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-12T18:20:41.103 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:20:41.103 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:20:41.103 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:20:41.103 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:20:41.103 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:20:41.212 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:20:41.212 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:20:41.213 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:20:41.214 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:20:41.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:20:41.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:20:41.215 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:20:41.216 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:20:41.216 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:20:41.216 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:20:41.217 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:20:41.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:20:41.218 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:20:41.218 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-12T18:20:41.233 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:20:41.234 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:41.234 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:41.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76821 2026-02-12T18:20:41.236 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:20:41.236 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:20:41.236 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-12T18:20:41.236 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-12T18:20:41.236 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-12T18:20:41.236 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-12T18:20:41.236 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-12T18:20:41.236 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:20:41.236 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-12T18:20:41.236 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:20:41.237 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:20:41.237 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:20:41.237 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:20:41.237 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:20:41.237 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:20:41.238 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:20:41.238 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:20:41.238 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:20:41.239 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:20:41.239 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:20:41.239 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:20:41.240 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:20:41.240 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:20:41.241 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:20:41.241 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:20:41.241 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:20:41.242 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:20:41.242 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-12T18:20:41.243 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:20:41.243 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:41.244 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:41.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76821 2026-02-12T18:20:41.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:20:41.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:20:41.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-12T18:20:41.246 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T18:20:41.246 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:41.246 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:41.246 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76821 2026-02-12T18:20:41.247 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T18:20:41.248 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T18:20:41.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T18:20:41.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-12T18:20:41.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-12T18:20:41.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-12T18:20:41.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T18:20:41.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T18:20:41.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T18:20:41.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-12T18:20:41.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-12T18:20:41.266 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T18:20:41.266 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:20:41.266 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:20:41.266 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:20:41.266 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:41.266 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:41.267 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:20:41.267 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T18:20:41.289 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T18:20:41.290 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T18:20:41.290 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:20:41.290 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:20:41.290 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T18:20:41.291 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T18:20:41.291 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:20:41.291 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:20:41.291 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:20:41.291 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:20:41.291 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:41.291 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:41.292 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:20:41.292 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:20:41.292 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76821/ceph-mon.a.asok config get fsid 2026-02-12T18:20:41.336 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T18:20:41.336 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:20:41.336 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:20:41.337 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T18:20:41.337 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T18:20:41.337 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:20:41.337 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:20:41.337 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:20:41.338 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:20:41.338 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:41.338 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:41.338 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:20:41.338 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:20:41.339 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76821/ceph-mon.a.asok config get mon_host 2026-02-12T18:20:41.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-12T18:20:41.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-12T18:20:41.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T18:20:41.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T18:20:41.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T18:20:41.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-12T18:20:41.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T18:20:41.479 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T18:20:41.479 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:20:41.480 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:20:41.480 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:20:41.480 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:41.480 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:41.480 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:20:41.481 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:20:41.481 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:20:41.496 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-12T18:20:41.497 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T18:20:41.592 INFO:tasks.workunit.client.0.trial116.stderr:pool 'rbd' already exists 2026-02-12T18:20:41.598 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:20:42.601 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-12T18:20:42.601 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:20:42.601 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:20:42.601 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:20:42.601 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:42.601 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:42.601 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:20:42.601 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-12T18:20:42.601 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-mon.a.asok log flush 2026-02-12T18:20:42.640 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:20:42.645 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-12T18:20:42.646 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:20:41.280+0000 7fd8aec38c00 0 load: jerasure load: lrc 2026-02-12T18:20:42.647 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_unfound td/test-erasure-eio 2026-02-12T18:20:42.647 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:629: TEST_ec_recovery_unfound: local dir=td/test-erasure-eio 2026-02-12T18:20:42.647 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:630: TEST_ec_recovery_unfound: local objname=myobject 2026-02-12T18:20:42.647 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:631: TEST_ec_recovery_unfound: local lastobj=100 2026-02-12T18:20:42.647 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:633: TEST_ec_recovery_unfound: local testobj=obj75 2026-02-12T18:20:42.647 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:635: TEST_ec_recovery_unfound: ORIG_ARGS='--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:20:42.647 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:636: TEST_ec_recovery_unfound: CEPH_ARGS+=' --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-12T18:20:42.647 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: CEPH_ARGS+=' --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-12T18:20:42.647 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:638: TEST_ec_recovery_unfound: setup_osds 5 2026-02-12T18:20:42.647 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=5 2026-02-12T18:20:42.647 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-12T18:20:42.647 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 5 - 1 2026-02-12T18:20:42.648 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 4 2026-02-12T18:20:42.649 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:20:42.650 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-12T18:20:42.650 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:20:42.650 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:20:42.650 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T18:20:42.650 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:20:42.650 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-12T18:20:42.650 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-12T18:20:42.650 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:20:42.650 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:20:42.650 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:20:42.650 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-12T18:20:42.650 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-12T18:20:42.650 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:20:42.650 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:20:42.650 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:20:42.650 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:20:42.650 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:20:42.650 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:20:42.650 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:20:42.650 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:42.650 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:42.650 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:20:42.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:20:42.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:20:42.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:20:42.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:20:42.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:20:42.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:20:42.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:20:42.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:20:42.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:20:42.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:20:42.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:20:42.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:20:42.652 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-12T18:20:42.652 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:20:42.652 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3a026eec-dbdb-40ed-8c1e-489f62071ff2 2026-02-12T18:20:42.652 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 3a026eec-dbdb-40ed-8c1e-489f62071ff2' 2026-02-12T18:20:42.652 INFO:tasks.workunit.client.0.trial116.stdout:add osd0 3a026eec-dbdb-40ed-8c1e-489f62071ff2 2026-02-12T18:20:42.652 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:20:42.665 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD6GY5pcu51JxAA7muaXaZKlnW1IqxawzmVGA== 2026-02-12T18:20:42.665 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD6GY5pcu51JxAA7muaXaZKlnW1IqxawzmVGA=="}' 2026-02-12T18:20:42.665 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3a026eec-dbdb-40ed-8c1e-489f62071ff2 -i td/test-erasure-eio/0/new.json 2026-02-12T18:20:42.770 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:20:42.777 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-12T18:20:42.778 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD6GY5pcu51JxAA7muaXaZKlnW1IqxawzmVGA== --osd-uuid 3a026eec-dbdb-40ed-8c1e-489f62071ff2 2026-02-12T18:20:42.798 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:42.794+0000 7f127e630780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:42.800 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:42.797+0000 7f127e630780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:42.801 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:42.797+0000 7f127e630780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:42.802 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:42.798+0000 7f127e630780 -1 bdev(0x3538c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-12T18:20:42.802 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:42.798+0000 7f127e630780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-12T18:20:43.204 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-12T18:20:43.205 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:20:43.205 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T18:20:43.206 INFO:tasks.workunit.client.0.trial116.stdout:adding osd0 key to auth repository 2026-02-12T18:20:43.206 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:20:43.313 INFO:tasks.workunit.client.0.trial116.stdout:start osd.0 2026-02-12T18:20:43.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T18:20:43.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:20:43.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:20:43.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:20:43.315 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:20:43.333 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:43.329+0000 7fd03cfd4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:43.334 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:43.331+0000 7fd03cfd4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:43.335 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:43.332+0000 7fd03cfd4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:43.413 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T18:20:43.413 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:20:43.413 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T18:20:43.413 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:20:43.413 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:20:43.413 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:20:43.414 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:20:43.414 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:20:43.414 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:20:43.414 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:20:43.513 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:20:43.533 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:43.530+0000 7fd03cfd4780 -1 Falling back to public interface 2026-02-12T18:20:43.819 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:43.815+0000 7fd03cfd4780 -1 osd.0 0 log_to_monitors true 2026-02-12T18:20:44.514 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:20:44.515 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:20:44.515 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:20:44.516 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:20:44.516 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:20:44.516 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:20:44.745 INFO:tasks.workunit.client.0.trial116.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3511896663,v1:127.0.0.1:6803/3511896663] [v2:127.0.0.1:6804/3511896663,v1:127.0.0.1:6805/3511896663] exists,up 3a026eec-dbdb-40ed-8c1e-489f62071ff2 2026-02-12T18:20:44.750 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:20:44.750 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:20:44.750 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:20:44.750 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:20:44.750 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-12T18:20:44.750 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:20:44.750 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:20:44.750 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-12T18:20:44.750 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:20:44.750 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-12T18:20:44.751 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-12T18:20:44.751 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:20:44.751 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:20:44.751 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:20:44.751 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-12T18:20:44.751 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-12T18:20:44.751 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:20:44.751 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:20:44.751 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:20:44.751 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:20:44.751 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:20:44.751 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:20:44.752 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:20:44.752 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:44.752 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:44.752 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:20:44.752 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:20:44.752 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:20:44.752 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:20:44.752 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:20:44.752 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:20:44.752 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:20:44.752 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:20:44.753 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:20:44.753 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:20:44.753 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:20:44.753 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:20:44.753 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:20:44.753 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-12T18:20:44.753 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:20:44.754 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=062cd0c8-0705-4aba-b217-ebd1aa42ef00 2026-02-12T18:20:44.754 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 062cd0c8-0705-4aba-b217-ebd1aa42ef00' 2026-02-12T18:20:44.754 INFO:tasks.workunit.client.0.trial116.stdout:add osd1 062cd0c8-0705-4aba-b217-ebd1aa42ef00 2026-02-12T18:20:44.754 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:20:44.762 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD8GY5peN9ELRAATOi2v6p/ERg+rjvzH+5NUA== 2026-02-12T18:20:44.762 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD8GY5peN9ELRAATOi2v6p/ERg+rjvzH+5NUA=="}' 2026-02-12T18:20:44.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 062cd0c8-0705-4aba-b217-ebd1aa42ef00 -i td/test-erasure-eio/1/new.json 2026-02-12T18:20:44.998 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:20:45.006 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-12T18:20:45.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD8GY5peN9ELRAATOi2v6p/ERg+rjvzH+5NUA== --osd-uuid 062cd0c8-0705-4aba-b217-ebd1aa42ef00 2026-02-12T18:20:45.027 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:45.023+0000 7f8128a15780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:45.028 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:45.025+0000 7f8128a15780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:45.029 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:45.025+0000 7f8128a15780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:45.030 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:45.026+0000 7f8128a15780 -1 bdev(0x2756c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-12T18:20:45.030 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:45.026+0000 7f8128a15780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-12T18:20:45.354 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-12T18:20:45.354 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:20:45.355 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-12T18:20:45.355 INFO:tasks.workunit.client.0.trial116.stdout:adding osd1 key to auth repository 2026-02-12T18:20:45.356 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:20:45.680 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-12T18:20:45.681 INFO:tasks.workunit.client.0.trial116.stdout:start osd.1 2026-02-12T18:20:45.682 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:20:45.682 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:20:45.682 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:20:45.682 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:20:45.699 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:45.695+0000 7f82b0487780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:45.700 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:45.697+0000 7f82b0487780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:45.701 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:45.697+0000 7f82b0487780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:45.837 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:45.832+0000 7f82b0487780 -1 Falling back to public interface 2026-02-12T18:20:45.909 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-12T18:20:45.909 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:20:45.909 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T18:20:45.909 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:20:45.910 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:20:45.910 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:20:45.910 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:20:45.911 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:20:45.911 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:20:45.911 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:20:46.116 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:46.112+0000 7f82b0487780 -1 osd.1 0 log_to_monitors true 2026-02-12T18:20:46.140 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:20:47.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:20:47.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:20:47.142 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:20:47.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:20:47.143 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:20:47.143 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:20:47.373 INFO:tasks.workunit.client.0.trial116.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/4285769679,v1:127.0.0.1:6811/4285769679] [v2:127.0.0.1:6812/4285769679,v1:127.0.0.1:6813/4285769679] exists,up 062cd0c8-0705-4aba-b217-ebd1aa42ef00 2026-02-12T18:20:47.377 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:20:47.378 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:20:47.378 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:20:47.378 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:20:47.378 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-12T18:20:47.378 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:20:47.378 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:20:47.378 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-12T18:20:47.378 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:20:47.378 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-12T18:20:47.378 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-12T18:20:47.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:20:47.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:20:47.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:20:47.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-12T18:20:47.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-12T18:20:47.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:20:47.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:20:47.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:20:47.379 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:20:47.379 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:20:47.379 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:20:47.380 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:20:47.380 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:47.380 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:47.380 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:20:47.380 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:20:47.380 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:20:47.380 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:20:47.380 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:20:47.380 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:20:47.380 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:20:47.380 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:20:47.380 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:20:47.380 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:20:47.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:20:47.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:20:47.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:20:47.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-12T18:20:47.381 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:20:47.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b2073d89-65aa-49db-838a-f88821e653f6 2026-02-12T18:20:47.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 b2073d89-65aa-49db-838a-f88821e653f6' 2026-02-12T18:20:47.381 INFO:tasks.workunit.client.0.trial116.stdout:add osd2 b2073d89-65aa-49db-838a-f88821e653f6 2026-02-12T18:20:47.382 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:20:47.390 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD/GY5pUOkNFxAAVBigWn0y8fFvFI+1x+Q4yA== 2026-02-12T18:20:47.390 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD/GY5pUOkNFxAAVBigWn0y8fFvFI+1x+Q4yA=="}' 2026-02-12T18:20:47.390 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b2073d89-65aa-49db-838a-f88821e653f6 -i td/test-erasure-eio/2/new.json 2026-02-12T18:20:47.622 INFO:tasks.workunit.client.0.trial116.stdout:2 2026-02-12T18:20:47.630 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-12T18:20:47.631 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD/GY5pUOkNFxAAVBigWn0y8fFvFI+1x+Q4yA== --osd-uuid b2073d89-65aa-49db-838a-f88821e653f6 2026-02-12T18:20:47.652 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:47.649+0000 7fa885055780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:47.653 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:47.650+0000 7fa885055780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:47.654 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:47.651+0000 7fa885055780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:47.655 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:47.652+0000 7fa885055780 -1 bdev(0x2dcac00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-12T18:20:47.655 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:47.652+0000 7fa885055780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-12T18:20:47.969 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-12T18:20:47.969 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:20:47.970 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-12T18:20:47.970 INFO:tasks.workunit.client.0.trial116.stdout:adding osd2 key to auth repository 2026-02-12T18:20:47.970 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:20:48.289 INFO:tasks.workunit.client.0.trial116.stdout:start osd.2 2026-02-12T18:20:48.290 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-12T18:20:48.290 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:20:48.290 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:20:48.290 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:20:48.290 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:20:48.307 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:48.304+0000 7f2ef4e30780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:48.308 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:48.306+0000 7f2ef4e30780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:48.309 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:48.307+0000 7f2ef4e30780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:48.515 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:48.513+0000 7f2ef4e30780 -1 Falling back to public interface 2026-02-12T18:20:48.519 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-12T18:20:48.519 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:20:48.519 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T18:20:48.519 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:20:48.519 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:20:48.520 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:20:48.520 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:20:48.520 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:20:48.520 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:20:48.521 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:20:48.752 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:20:48.821 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:48.819+0000 7f2ef4e30780 -1 osd.2 0 log_to_monitors true 2026-02-12T18:20:49.753 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:20:49.754 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:20:49.754 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:20:49.755 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:20:49.755 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:20:49.755 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:20:49.982 INFO:tasks.workunit.client.0.trial116.stdout:osd.2 up in weight 1 up_from 19 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1395858714,v1:127.0.0.1:6819/1395858714] [v2:127.0.0.1:6820/1395858714,v1:127.0.0.1:6821/1395858714] exists,up b2073d89-65aa-49db-838a-f88821e653f6 2026-02-12T18:20:49.987 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:20:49.987 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:20:49.987 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:20:49.987 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:20:49.987 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-12T18:20:49.987 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:20:49.987 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:20:49.987 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-12T18:20:49.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:20:49.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-12T18:20:49.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-12T18:20:49.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:20:49.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:20:49.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:20:49.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-12T18:20:49.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-12T18:20:49.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:20:49.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:20:49.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:20:49.989 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:20:49.989 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:20:49.989 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:20:49.989 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:20:49.989 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:49.989 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:49.989 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:20:49.989 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:20:49.989 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:20:49.989 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:20:49.990 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:20:49.990 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:20:49.990 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:20:49.990 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:20:49.990 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:20:49.990 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:20:49.990 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:20:49.990 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:20:49.990 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:20:49.991 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-12T18:20:49.991 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:20:49.991 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=50a10149-6e29-4b7d-8490-c899392ebc5b 2026-02-12T18:20:49.991 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 50a10149-6e29-4b7d-8490-c899392ebc5b' 2026-02-12T18:20:49.991 INFO:tasks.workunit.client.0.trial116.stdout:add osd3 50a10149-6e29-4b7d-8490-c899392ebc5b 2026-02-12T18:20:49.991 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:20:49.999 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQABGo5pb39gOxAA6V52Glgj3gLokiETwzawWQ== 2026-02-12T18:20:50.000 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQABGo5pb39gOxAA6V52Glgj3gLokiETwzawWQ=="}' 2026-02-12T18:20:50.000 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 50a10149-6e29-4b7d-8490-c899392ebc5b -i td/test-erasure-eio/3/new.json 2026-02-12T18:20:50.234 INFO:tasks.workunit.client.0.trial116.stdout:3 2026-02-12T18:20:50.241 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-12T18:20:50.242 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQABGo5pb39gOxAA6V52Glgj3gLokiETwzawWQ== --osd-uuid 50a10149-6e29-4b7d-8490-c899392ebc5b 2026-02-12T18:20:50.263 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:50.260+0000 7f18d893d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:50.264 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:50.262+0000 7f18d893d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:50.265 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:50.263+0000 7f18d893d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:50.266 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:50.263+0000 7f18d893d780 -1 bdev(0x42d8c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-12T18:20:50.266 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:50.263+0000 7f18d893d780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-12T18:20:50.584 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-12T18:20:50.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:20:50.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-12T18:20:50.586 INFO:tasks.workunit.client.0.trial116.stdout:adding osd3 key to auth repository 2026-02-12T18:20:50.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:20:50.910 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-12T18:20:50.911 INFO:tasks.workunit.client.0.trial116.stdout:start osd.3 2026-02-12T18:20:50.911 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:20:50.912 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:20:50.912 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:20:50.912 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:20:50.929 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:50.926+0000 7fbe7b829780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:50.930 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:50.927+0000 7fbe7b829780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:50.931 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:50.928+0000 7fbe7b829780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:51.127 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:51.125+0000 7fbe7b829780 -1 Falling back to public interface 2026-02-12T18:20:51.140 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-12T18:20:51.140 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:20:51.140 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-12T18:20:51.140 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:20:51.140 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:20:51.140 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:20:51.141 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:20:51.141 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:20:51.141 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:20:51.141 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:20:51.377 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:20:51.461 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:51.458+0000 7fbe7b829780 -1 osd.3 0 log_to_monitors true 2026-02-12T18:20:52.378 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:20:52.378 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:20:52.378 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:20:52.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:20:52.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:20:52.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:20:52.616 INFO:tasks.workunit.client.0.trial116.stdout:osd.3 up in weight 1 up_from 26 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1536720877,v1:127.0.0.1:6827/1536720877] [v2:127.0.0.1:6828/1536720877,v1:127.0.0.1:6829/1536720877] exists,up 50a10149-6e29-4b7d-8490-c899392ebc5b 2026-02-12T18:20:52.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:20:52.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:20:52.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:20:52.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:20:52.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-12T18:20:52.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:20:52.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:20:52.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-12T18:20:52.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:20:52.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-12T18:20:52.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-12T18:20:52.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:20:52.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:20:52.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:20:52.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-12T18:20:52.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-12T18:20:52.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:20:52.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:20:52.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:20:52.622 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:20:52.622 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:20:52.622 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:20:52.622 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:20:52.622 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:52.622 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:52.622 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:20:52.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:20:52.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:20:52.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:20:52.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:20:52.623 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:20:52.623 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:20:52.623 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:20:52.623 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:20:52.623 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:20:52.623 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:20:52.623 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:20:52.623 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:20:52.623 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-12T18:20:52.623 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:20:52.623 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a82eaf60-9aae-48e0-98c7-6abc43afc749 2026-02-12T18:20:52.623 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 a82eaf60-9aae-48e0-98c7-6abc43afc749' 2026-02-12T18:20:52.624 INFO:tasks.workunit.client.0.trial116.stdout:add osd4 a82eaf60-9aae-48e0-98c7-6abc43afc749 2026-02-12T18:20:52.624 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:20:52.632 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAEGo5pmO2CJRAAgfTDc3oKlG6forxZeaQ5SQ== 2026-02-12T18:20:52.632 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAEGo5pmO2CJRAAgfTDc3oKlG6forxZeaQ5SQ=="}' 2026-02-12T18:20:52.632 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a82eaf60-9aae-48e0-98c7-6abc43afc749 -i td/test-erasure-eio/4/new.json 2026-02-12T18:20:52.875 INFO:tasks.workunit.client.0.trial116.stdout:4 2026-02-12T18:20:52.884 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-12T18:20:52.884 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAEGo5pmO2CJRAAgfTDc3oKlG6forxZeaQ5SQ== --osd-uuid a82eaf60-9aae-48e0-98c7-6abc43afc749 2026-02-12T18:20:52.904 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:52.902+0000 7fec40030780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:52.906 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:52.903+0000 7fec40030780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:52.907 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:52.904+0000 7fec40030780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:52.907 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:52.904+0000 7fec40030780 -1 bdev(0x2a2cc00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-12T18:20:52.907 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:52.904+0000 7fec40030780 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-12T18:20:53.241 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-12T18:20:53.241 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:20:53.242 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-12T18:20:53.242 INFO:tasks.workunit.client.0.trial116.stdout:adding osd4 key to auth repository 2026-02-12T18:20:53.242 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:20:53.576 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-12T18:20:53.576 INFO:tasks.workunit.client.0.trial116.stdout:start osd.4 2026-02-12T18:20:53.577 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:20:53.577 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:20:53.577 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:20:53.578 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:20:53.596 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:53.593+0000 7ff18423d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:53.597 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:53.594+0000 7ff18423d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:53.598 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:53.595+0000 7ff18423d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:20:53.719 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:53.716+0000 7ff18423d780 -1 Falling back to public interface 2026-02-12T18:20:53.803 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-12T18:20:53.804 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:20:53.804 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-12T18:20:53.804 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:20:53.804 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:20:53.804 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:20:53.804 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:20:53.804 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:20:53.805 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:20:53.805 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-12T18:20:54.035 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:20:54.095 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:20:54.092+0000 7ff18423d780 -1 osd.4 0 log_to_monitors true 2026-02-12T18:20:55.036 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:20:55.037 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:20:55.037 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:20:55.038 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:20:55.038 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:20:55.038 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-12T18:20:55.270 INFO:tasks.workunit.client.0.trial116.stdout:osd.4 up in weight 1 up_from 35 up_thru 36 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/455210439,v1:127.0.0.1:6835/455210439] [v2:127.0.0.1:6836/455210439,v1:127.0.0.1:6837/455210439] exists,up a82eaf60-9aae-48e0-98c7-6abc43afc749 2026-02-12T18:20:55.271 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:20:55.271 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:20:55.272 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:20:55.272 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-12T18:20:55.272 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T18:20:55.272 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T18:20:55.272 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:20:55.272 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:20:55.272 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:20:55.272 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.0.asok 2026-02-12T18:20:55.272 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-12T18:20:55.272 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-osd.0.asok log flush 2026-02-12T18:20:55.317 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:20:55.322 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-12T18:20:55.325 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:20:43.532+0000 7fd03cfd4780 0 load: jerasure load: lrc 2026-02-12T18:20:55.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: CEPH_ARGS='--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:20:55.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:641: TEST_ec_recovery_unfound: local poolname=pool-jerasure 2026-02-12T18:20:55.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:642: TEST_ec_recovery_unfound: create_erasure_coded_pool pool-jerasure 3 2 2026-02-12T18:20:55.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-12T18:20:55.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-12T18:20:55.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-12T18:20:55.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-12T18:20:55.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-12T18:20:55.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-12T18:20:55.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-12T18:20:55.613 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-12T18:20:55.613 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-12T18:20:55.987 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-jerasure' already exists 2026-02-12T18:20:55.994 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:20:56.996 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-12T18:20:56.996 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:20:56.997 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:20:56.997 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:20:56.997 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:20:56.998 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:20:56.998 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:20:56.998 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:20:56.998 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:20:56.998 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:20:57.083 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:20:57.083 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:20:57.083 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:20:57.083 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:20:57.083 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:20:57.084 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:20:57.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:20:57.314 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:20:57.314 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:20:57.314 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:20:57.314 INFO:tasks.workunit.client.0.trial116.stderr:4' 2026-02-12T18:20:57.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:20:57.315 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:20:57.315 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:20:57.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803781 2026-02-12T18:20:57.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803781 2026-02-12T18:20:57.380 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781' 2026-02-12T18:20:57.380 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:20:57.380 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:20:57.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574854 2026-02-12T18:20:57.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574854 2026-02-12T18:20:57.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-55834574854' 2026-02-12T18:20:57.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:20:57.443 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:20:57.510 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378628 2026-02-12T18:20:57.510 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378628 2026-02-12T18:20:57.510 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-55834574854 2-81604378628' 2026-02-12T18:20:57.510 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:20:57.511 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:20:57.574 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149699 2026-02-12T18:20:57.575 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149699 2026-02-12T18:20:57.575 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-55834574854 2-81604378628 3-111669149699' 2026-02-12T18:20:57.575 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:20:57.575 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-12T18:20:57.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855363 2026-02-12T18:20:57.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855363 2026-02-12T18:20:57.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-55834574854 2-81604378628 3-111669149699 4-150323855363' 2026-02-12T18:20:57.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:20:57.639 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803781 2026-02-12T18:20:57.639 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:20:57.639 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:20:57.640 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803781 2026-02-12T18:20:57.640 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:20:57.641 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803781 2026-02-12T18:20:57.641 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-12T18:20:57.641 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.0 seq 25769803781 2026-02-12T18:20:57.642 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:20:57.866 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803781 2026-02-12T18:20:57.866 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:20:57.867 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574854 2026-02-12T18:20:57.867 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:20:57.867 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:20:57.868 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574854 2026-02-12T18:20:57.868 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:20:57.869 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574854 2026-02-12T18:20:57.869 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574854' 2026-02-12T18:20:57.869 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.1 seq 55834574854 2026-02-12T18:20:57.870 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:20:58.096 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574854 -lt 55834574854 2026-02-12T18:20:58.096 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:20:58.096 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-81604378628 2026-02-12T18:20:58.096 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:20:58.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:20:58.098 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-81604378628 2026-02-12T18:20:58.098 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:20:58.099 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=81604378628 2026-02-12T18:20:58.099 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 81604378628' 2026-02-12T18:20:58.099 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.2 seq 81604378628 2026-02-12T18:20:58.099 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:20:58.325 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 81604378626 -lt 81604378628 2026-02-12T18:20:58.325 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:20:59.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:20:59.327 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:20:59.553 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 81604378628 -lt 81604378628 2026-02-12T18:20:59.553 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:20:59.553 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149699 2026-02-12T18:20:59.553 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:20:59.554 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:20:59.555 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149699 2026-02-12T18:20:59.555 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:20:59.556 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149699 2026-02-12T18:20:59.556 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149699' 2026-02-12T18:20:59.556 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.3 seq 111669149699 2026-02-12T18:20:59.556 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:20:59.781 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149700 -lt 111669149699 2026-02-12T18:20:59.781 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:20:59.782 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-150323855363 2026-02-12T18:20:59.782 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:20:59.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-12T18:20:59.784 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-150323855363 2026-02-12T18:20:59.784 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:20:59.784 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=150323855363 2026-02-12T18:20:59.784 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 150323855363' 2026-02-12T18:20:59.785 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.4 seq 150323855363 2026-02-12T18:20:59.785 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-12T18:21:00.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 150323855363 -lt 150323855363 2026-02-12T18:21:00.010 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:21:00.010 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:21:00.010 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:21:00.325 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:21:00.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:21:00.326 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:21:00.326 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:21:00.326 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:21:00.326 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:21:00.326 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:21:00.326 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:21:00.562 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:21:00.562 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:21:00.562 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:21:00.563 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:21:00.875 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:21:00.875 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:21:00.875 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:21:00.875 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:644: TEST_ec_recovery_unfound: ceph pg dump pgs 2026-02-12T18:21:01.099 INFO:tasks.workunit.client.0.trial116.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-12T18:21:01.100 INFO:tasks.workunit.client.0.trial116.stdout:2.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-12T18:20:57.096433+0000 0'0 42:16 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-12T18:20:55.921870+0000 0'0 2026-02-12T18:20:55.921870+0000 0 0 periodic scrub scheduled @ 2026-02-13T23:39:31.965044+0000 0 0 2026-02-12T18:21:01.100 INFO:tasks.workunit.client.0.trial116.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-12T18:20:54.987504+0000 0'0 42:20 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-12T18:20:41.571345+0000 0'0 2026-02-12T18:20:41.571345+0000 0 0 periodic scrub scheduled @ 2026-02-14T04:42:37.421262+0000 0 0 2026-02-12T18:21:01.100 INFO:tasks.workunit.client.0.trial116.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-12T18:20:54.987535+0000 0'0 42:100 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-12T18:20:41.571345+0000 0'0 2026-02-12T18:20:41.571345+0000 0 0 periodic scrub scheduled @ 2026-02-13T23:15:50.322151+0000 0 0 2026-02-12T18:21:01.100 INFO:tasks.workunit.client.0.trial116.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-12T18:20:55.487769+0000 0'0 42:20 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-12T18:20:41.571345+0000 0'0 2026-02-12T18:20:41.571345+0000 0 0 periodic scrub scheduled @ 2026-02-14T01:59:11.015466+0000 0 0 2026-02-12T18:21:01.100 INFO:tasks.workunit.client.0.trial116.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-12T18:20:49.558404+0000 0'0 42:65 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-12T18:20:41.571345+0000 0'0 2026-02-12T18:20:41.571345+0000 0 0 periodic scrub scheduled @ 2026-02-13T19:14:46.358433+0000 0 0 2026-02-12T18:21:01.100 INFO:tasks.workunit.client.0.trial116.stdout: 2026-02-12T18:21:01.100 INFO:tasks.workunit.client.0.trial116.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-12T18:21:01.101 INFO:tasks.workunit.client.0.trial116.stderr:dumped pgs 2026-02-12T18:21:01.107 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:646: TEST_ec_recovery_unfound: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-12T18:21:01.108 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-12T18:21:01.108 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-12T18:21:01.108 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-12T18:21:01.108 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:21:01.108 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-12T18:21:01.108 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:21:01.108 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-12T18:21:01.108 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:21:01.108 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-12T18:21:01.108 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:21:01.109 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-12T18:21:01.109 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:01.126 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:648: TEST_ec_recovery_unfound: get_osds pool-jerasure myobject 2026-02-12T18:21:01.127 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:21:01.127 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-12T18:21:01.127 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-12T18:21:01.127 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:21:01.360 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:21:01.360 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:21:01.360 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:21:01.360 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:21:01.360 INFO:tasks.workunit.client.0.trial116.stderr:2' 2026-02-12T18:21:01.360 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2 2026-02-12T18:21:01.361 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:648: TEST_ec_recovery_unfound: initial_osds=('3' '1' '4' '0' '2') 2026-02-12T18:21:01.361 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:648: TEST_ec_recovery_unfound: local -a initial_osds 2026-02-12T18:21:01.361 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:649: TEST_ec_recovery_unfound: local last_osd=2 2026-02-12T18:21:01.361 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:650: TEST_ec_recovery_unfound: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-12T18:21:01.361 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:21:01.361 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:21:01.361 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:21:01.361 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:21:01.361 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:21:01.666 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:21:01.666 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:651: TEST_ec_recovery_unfound: ceph osd down 2 2026-02-12T18:21:01.908 INFO:tasks.workunit.client.0.trial116.stderr:osd.2 is already down. 2026-02-12T18:21:01.915 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:652: TEST_ec_recovery_unfound: ceph osd out 2 2026-02-12T18:21:02.157 INFO:tasks.workunit.client.0.trial116.stderr:osd.2 is already out. 2026-02-12T18:21:02.165 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:654: TEST_ec_recovery_unfound: ceph pg dump pgs 2026-02-12T18:21:02.382 INFO:tasks.workunit.client.0.trial116.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-12T18:21:02.383 INFO:tasks.workunit.client.0.trial116.stdout:2.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-12T18:20:57.096433+0000 0'0 42:16 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-12T18:20:55.921870+0000 0'0 2026-02-12T18:20:55.921870+0000 0 0 periodic scrub scheduled @ 2026-02-13T23:39:31.965044+0000 0 0 2026-02-12T18:21:02.383 INFO:tasks.workunit.client.0.trial116.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-12T18:20:54.987504+0000 0'0 42:20 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-12T18:20:41.571345+0000 0'0 2026-02-12T18:20:41.571345+0000 0 0 periodic scrub scheduled @ 2026-02-14T04:42:37.421262+0000 0 0 2026-02-12T18:21:02.383 INFO:tasks.workunit.client.0.trial116.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-12T18:20:54.987535+0000 0'0 42:100 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-12T18:20:41.571345+0000 0'0 2026-02-12T18:20:41.571345+0000 0 0 periodic scrub scheduled @ 2026-02-13T23:15:50.322151+0000 0 0 2026-02-12T18:21:02.383 INFO:tasks.workunit.client.0.trial116.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-12T18:20:55.487769+0000 0'0 42:20 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-12T18:20:41.571345+0000 0'0 2026-02-12T18:20:41.571345+0000 0 0 periodic scrub scheduled @ 2026-02-14T01:59:11.015466+0000 0 0 2026-02-12T18:21:02.383 INFO:tasks.workunit.client.0.trial116.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 peering 2026-02-12T18:21:02.195773+0000 0'0 44:74 [1,0,4] 1 [1,0,4] 1 0'0 2026-02-12T18:20:41.571345+0000 0'0 2026-02-12T18:20:41.571345+0000 0 0 no scrub is scheduled 0 0 2026-02-12T18:21:02.383 INFO:tasks.workunit.client.0.trial116.stdout: 2026-02-12T18:21:02.383 INFO:tasks.workunit.client.0.trial116.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-12T18:21:02.384 INFO:tasks.workunit.client.0.trial116.stderr:dumped pgs 2026-02-12T18:21:02.390 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:656: TEST_ec_recovery_unfound: dd if=/dev/urandom of=td/test-erasure-eio/ORIGINAL bs=1024 count=4 2026-02-12T18:21:02.391 INFO:tasks.workunit.client.0.trial116.stderr:4+0 records in 2026-02-12T18:21:02.391 INFO:tasks.workunit.client.0.trial116.stderr:4+0 records out 2026-02-12T18:21:02.391 INFO:tasks.workunit.client.0.trial116.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 8.2203e-05 s, 49.8 MB/s 2026-02-12T18:21:02.391 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: seq 1 100 2026-02-12T18:21:02.392 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.392 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj1 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.411 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.412 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj2 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.429 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.429 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj3 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.446 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.446 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj4 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.463 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.463 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj5 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.480 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.480 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj6 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.497 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.498 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj7 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.515 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.515 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj8 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.531 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.531 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj9 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.548 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.548 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj10 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.566 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.566 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj11 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.582 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.583 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj12 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.599 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.599 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj13 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.616 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.616 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj14 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.634 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.634 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj15 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.650 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj16 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.667 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.667 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj17 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.684 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.684 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj18 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.703 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.703 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj19 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.720 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.720 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj20 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.738 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.738 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj21 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.755 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.755 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj22 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.772 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.772 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj23 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.789 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.789 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj24 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.805 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.805 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj25 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.822 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.822 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj26 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.839 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.839 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj27 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.856 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.856 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj28 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.873 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.873 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj29 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.889 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.889 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj30 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj31 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.923 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.923 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj32 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj33 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.957 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.957 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj34 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.974 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.974 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj35 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:02.991 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:02.991 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj36 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj37 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.024 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.024 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj38 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.041 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.041 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj39 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.058 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.058 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj40 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.074 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.074 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj41 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.091 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.092 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj42 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.108 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.108 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj43 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.125 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.125 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj44 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.141 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj45 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.157 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.157 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj46 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.174 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.174 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj47 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.192 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.192 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj48 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.209 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.209 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj49 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.226 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.226 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj50 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.243 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.243 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj51 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.260 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.260 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj52 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.276 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.276 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj53 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.293 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.293 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj54 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.310 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.310 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj55 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.327 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.327 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj56 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.344 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.345 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj57 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.361 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.361 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj58 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.378 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.378 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj59 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.395 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.395 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj60 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.411 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.411 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj61 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.428 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.428 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj62 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj63 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.461 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.461 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj64 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.479 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.479 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj65 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.496 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.496 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj66 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.512 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.512 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj67 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.529 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.529 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj68 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.546 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.547 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj69 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.563 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.563 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj70 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.580 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.580 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj71 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.597 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.597 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj72 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.614 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.614 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj73 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.630 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.630 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj74 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.647 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.647 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj75 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj76 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.681 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.681 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj77 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.697 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.697 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj78 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.714 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.714 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj79 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.732 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj80 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj81 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.766 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.766 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj82 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj83 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.799 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.799 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj84 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.816 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.816 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj85 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.832 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.832 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj86 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.849 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.849 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj87 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.865 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.865 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj88 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.882 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.882 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj89 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.898 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.898 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj90 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.915 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.915 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj91 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.931 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.931 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj92 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.948 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.948 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj93 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.964 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.965 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj94 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.981 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.981 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj95 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:03.998 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:03.998 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj96 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:04.015 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:04.015 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj97 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:04.033 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:04.033 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj98 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:04.050 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:04.050 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj99 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:04.066 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:04.066 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj100 td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:04.084 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:662: TEST_ec_recovery_unfound: inject_eio ec data pool-jerasure obj75 td/test-erasure-eio 0 2026-02-12T18:21:04.084 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-12T18:21:04.084 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-12T18:21:04.084 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-12T18:21:04.085 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-12T18:21:04.085 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-12T18:21:04.085 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-12T18:21:04.085 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj75 2026-02-12T18:21:04.085 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-12T18:21:04.085 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-12T18:21:04.085 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-12T18:21:04.085 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=0 2026-02-12T18:21:04.085 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-12T18:21:04.085 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj75 2026-02-12T18:21:04.085 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:21:04.085 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj75 2026-02-12T18:21:04.086 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj75 2026-02-12T18:21:04.086 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:21:04.317 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:21:04.317 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:21:04.317 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:21:04.317 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:21:04.317 INFO:tasks.workunit.client.0.trial116.stderr:2147483647' 2026-02-12T18:21:04.318 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-12T18:21:04.318 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-12T18:21:04.318 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-12T18:21:04.318 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=3 2026-02-12T18:21:04.318 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-12T18:21:04.318 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/3/type 2026-02-12T18:21:04.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-12T18:21:04.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-12T18:21:04.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-12T18:21:04.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-12T18:21:04.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-12T18:21:04.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-12T18:21:04.320 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-12T18:21:04.320 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-12T18:21:04.321 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-12T18:21:04.321 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-12T18:21:04.321 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:21:04.321 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:21:04.321 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:21:04.321 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.3.asok 2026-02-12T18:21:04.321 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.76821/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-12T18:21:04.367 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-12T18:21:04.367 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-12T18:21:04.368 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-12T18:21:04.369 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.3 2026-02-12T18:21:04.369 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-12T18:21:04.369 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-12T18:21:04.369 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:21:04.369 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:21:04.369 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:21:04.369 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.3.asok 2026-02-12T18:21:04.369 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-12T18:21:04.369 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-osd.3.asok injectdataerr pool-jerasure obj75 0 2026-02-12T18:21:04.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:663: TEST_ec_recovery_unfound: inject_eio ec data pool-jerasure obj75 td/test-erasure-eio 1 2026-02-12T18:21:04.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-12T18:21:04.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-12T18:21:04.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-12T18:21:04.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-12T18:21:04.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-12T18:21:04.418 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-12T18:21:04.418 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj75 2026-02-12T18:21:04.418 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-12T18:21:04.418 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-12T18:21:04.418 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-12T18:21:04.418 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=1 2026-02-12T18:21:04.418 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-12T18:21:04.418 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj75 2026-02-12T18:21:04.418 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:21:04.418 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj75 2026-02-12T18:21:04.418 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:21:04.418 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj75 2026-02-12T18:21:04.645 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:21:04.646 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:21:04.646 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:21:04.646 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:21:04.646 INFO:tasks.workunit.client.0.trial116.stderr:2147483647' 2026-02-12T18:21:04.646 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-12T18:21:04.646 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-12T18:21:04.646 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-12T18:21:04.646 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=1 2026-02-12T18:21:04.646 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-12T18:21:04.647 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/1/type 2026-02-12T18:21:04.649 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-12T18:21:04.649 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-12T18:21:04.649 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-12T18:21:04.649 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-12T18:21:04.649 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-12T18:21:04.649 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-12T18:21:04.649 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-12T18:21:04.649 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-12T18:21:04.649 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-12T18:21:04.650 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-12T18:21:04.650 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:21:04.650 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:21:04.650 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:21:04.650 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.1.asok 2026-02-12T18:21:04.650 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.76821/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-12T18:21:04.696 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-12T18:21:04.696 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-12T18:21:04.698 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-12T18:21:04.698 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.1 2026-02-12T18:21:04.698 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-12T18:21:04.698 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-12T18:21:04.698 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:21:04.698 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:21:04.698 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:21:04.698 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.1.asok 2026-02-12T18:21:04.699 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-12T18:21:04.699 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-osd.1.asok injectdataerr pool-jerasure obj75 1 2026-02-12T18:21:04.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:665: TEST_ec_recovery_unfound: activate_osd td/test-erasure-eio 2 2026-02-12T18:21:04.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-12T18:21:04.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T18:21:04.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-12T18:21:04.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T18:21:04.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-12T18:21:04.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:21:04.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:21:04.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:21:04.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:21:04.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-12T18:21:04.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-12T18:21:04.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T18:21:04.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T18:21:04.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:21:04.750 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T18:21:04.750 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:21:04.750 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:21:04.750 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:21:04.750 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:21:04.750 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:21:04.750 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:21:04.750 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:21:04.750 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:21:04.751 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:21:04.751 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:21:04.751 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:21:04.751 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:21:04.751 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:21:04.751 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:21:04.751 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T18:21:04.751 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-12T18:21:04.751 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-12T18:21:04.751 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-12T18:21:04.752 INFO:tasks.workunit.client.0.trial116.stdout:start osd.2 2026-02-12T18:21:04.753 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:21:04.753 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-12T18:21:04.753 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-12T18:21:04.753 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T18:21:04.753 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T18:21:04.753 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T18:21:04.761 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:04.757+0000 7f57759fd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:04.762 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:04.758+0000 7f57759fd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:04.763 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:04.759+0000 7f57759fd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:04.893 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:04.889+0000 7f57759fd780 -1 Falling back to public interface 2026-02-12T18:21:04.971 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-12T18:21:04.971 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:21:04.971 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T18:21:04.971 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:21:04.972 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:21:04.972 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:21:04.972 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:21:04.972 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:21:04.973 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:21:04.973 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:21:05.198 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:21:05.226 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:05.222+0000 7f57759fd780 -1 osd.2 42 log_to_monitors true 2026-02-12T18:21:06.199 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:21:06.199 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:21:06.199 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:21:06.200 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:21:06.200 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:21:06.200 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:21:06.426 INFO:tasks.workunit.client.0.trial116.stdout:osd.2 up out weight 0 up_from 48 up_thru 19 down_at 43 last_clean_interval [19,42) [v2:127.0.0.1:6818/2140568493,v1:127.0.0.1:6819/2140568493] [v2:127.0.0.1:6820/2140568493,v1:127.0.0.1:6821/2140568493] exists,up b2073d89-65aa-49db-838a-f88821e653f6 2026-02-12T18:21:06.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:21:06.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:21:06.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:21:06.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:666: TEST_ec_recovery_unfound: ceph osd in 2 2026-02-12T18:21:06.666 INFO:tasks.workunit.client.0.trial116.stderr:osd.2 is already in. 2026-02-12T18:21:06.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:668: TEST_ec_recovery_unfound: sleep 15 2026-02-12T18:21:21.676 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:670: TEST_ec_recovery_unfound: seq 1 100 2026-02-12T18:21:21.677 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:670: TEST_ec_recovery_unfound: for tmp in $(seq 1 100) 2026-02-12T18:21:21.678 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:671: TEST_ec_recovery_unfound: get_state 2.0 2026-02-12T18:21:21.678 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:60: get_state: local pgid=2.0 2026-02-12T18:21:21.678 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:61: get_state: local sname=state 2026-02-12T18:21:21.678 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:62: get_state: ceph --format json pg dump pgs 2026-02-12T18:21:21.678 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:63: get_state: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .state' 2026-02-12T18:21:21.898 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:671: TEST_ec_recovery_unfound: state=active+recovery_unfound+degraded 2026-02-12T18:21:21.898 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:672: TEST_ec_recovery_unfound: echo active+recovery_unfound+degraded 2026-02-12T18:21:21.898 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:672: TEST_ec_recovery_unfound: grep recovery_unfound 2026-02-12T18:21:21.899 INFO:tasks.workunit.client.0.trial116.stdout:active+recovery_unfound+degraded 2026-02-12T18:21:21.899 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:673: TEST_ec_recovery_unfound: '[' 0 = 0 ']' 2026-02-12T18:21:21.899 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:674: TEST_ec_recovery_unfound: break 2026-02-12T18:21:21.899 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: ceph pg dump pgs 2026-02-12T18:21:22.113 INFO:tasks.workunit.client.0.trial116.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-12T18:21:22.113 INFO:tasks.workunit.client.0.trial116.stdout:2.0 101 1 3 0 1 413696 0 0 101 0 101 active+recovery_unfound+degraded 2026-02-12T18:21:06.703050+0000 46'101 53:362 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-12T18:20:55.921870+0000 0'0 2026-02-12T18:20:55.921870+0000 0 0 no scrub is scheduled 0 0 2026-02-12T18:21:22.113 INFO:tasks.workunit.client.0.trial116.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-12T18:21:06.701534+0000 0'0 53:60 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-12T18:20:41.571345+0000 0'0 2026-02-12T18:20:41.571345+0000 0 0 periodic scrub scheduled @ 2026-02-13T22:07:25.953685+0000 0 0 2026-02-12T18:21:22.114 INFO:tasks.workunit.client.0.trial116.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-12T18:20:54.987535+0000 0'0 53:119 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-12T18:20:41.571345+0000 0'0 2026-02-12T18:20:41.571345+0000 0 0 periodic scrub scheduled @ 2026-02-13T23:15:50.322151+0000 0 0 2026-02-12T18:21:22.114 INFO:tasks.workunit.client.0.trial116.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-12T18:20:55.487769+0000 0'0 53:42 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-12T18:20:41.571345+0000 0'0 2026-02-12T18:20:41.571345+0000 0 0 periodic scrub scheduled @ 2026-02-14T01:59:11.015466+0000 0 0 2026-02-12T18:21:22.114 INFO:tasks.workunit.client.0.trial116.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-12T18:21:06.701572+0000 0'0 53:104 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-12T18:20:41.571345+0000 0'0 2026-02-12T18:20:41.571345+0000 0 0 periodic scrub scheduled @ 2026-02-13T18:28:18.336962+0000 0 0 2026-02-12T18:21:22.114 INFO:tasks.workunit.client.0.trial116.stdout: 2026-02-12T18:21:22.114 INFO:tasks.workunit.client.0.trial116.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-12T18:21:22.114 INFO:tasks.workunit.client.0.trial116.stderr:dumped pgs 2026-02-12T18:21:22.120 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:681: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-12T18:21:22.181 INFO:tasks.workunit.client.0.trial116.stdout:{ 2026-02-12T18:21:22.181 INFO:tasks.workunit.client.0.trial116.stdout: "num_missing": 1, 2026-02-12T18:21:22.181 INFO:tasks.workunit.client.0.trial116.stdout: "num_unfound": 1, 2026-02-12T18:21:22.181 INFO:tasks.workunit.client.0.trial116.stdout: "objects": [ 2026-02-12T18:21:22.181 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:21:22.182 INFO:tasks.workunit.client.0.trial116.stdout: "oid": { 2026-02-12T18:21:22.182 INFO:tasks.workunit.client.0.trial116.stdout: "oid": "obj75", 2026-02-12T18:21:22.182 INFO:tasks.workunit.client.0.trial116.stdout: "key": "", 2026-02-12T18:21:22.182 INFO:tasks.workunit.client.0.trial116.stdout: "snapid": -2, 2026-02-12T18:21:22.182 INFO:tasks.workunit.client.0.trial116.stdout: "hash": 3565005477, 2026-02-12T18:21:22.182 INFO:tasks.workunit.client.0.trial116.stdout: "max": 0, 2026-02-12T18:21:22.182 INFO:tasks.workunit.client.0.trial116.stdout: "pool": 2, 2026-02-12T18:21:22.182 INFO:tasks.workunit.client.0.trial116.stdout: "namespace": "" 2026-02-12T18:21:22.182 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:21:22.182 INFO:tasks.workunit.client.0.trial116.stdout: "need": "46'76", 2026-02-12T18:21:22.182 INFO:tasks.workunit.client.0.trial116.stdout: "have": "0'0", 2026-02-12T18:21:22.182 INFO:tasks.workunit.client.0.trial116.stdout: "flags": "none", 2026-02-12T18:21:22.183 INFO:tasks.workunit.client.0.trial116.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2026-02-12T18:21:22.183 INFO:tasks.workunit.client.0.trial116.stdout: "locations": [ 2026-02-12T18:21:22.183 INFO:tasks.workunit.client.0.trial116.stdout: "0(3)", 2026-02-12T18:21:22.183 INFO:tasks.workunit.client.0.trial116.stdout: "4(2)" 2026-02-12T18:21:22.183 INFO:tasks.workunit.client.0.trial116.stdout: ] 2026-02-12T18:21:22.183 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:21:22.183 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:21:22.183 INFO:tasks.workunit.client.0.trial116.stdout: "state": "NotRecovering", 2026-02-12T18:21:22.183 INFO:tasks.workunit.client.0.trial116.stdout: "available_might_have_unfound": true, 2026-02-12T18:21:22.183 INFO:tasks.workunit.client.0.trial116.stdout: "might_have_unfound": [], 2026-02-12T18:21:22.183 INFO:tasks.workunit.client.0.trial116.stdout: "more": false 2026-02-12T18:21:22.183 INFO:tasks.workunit.client.0.trial116.stdout:} 2026-02-12T18:21:22.186 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: ceph pg 2.0 query 2026-02-12T18:21:22.248 INFO:tasks.workunit.client.0.trial116.stdout:{ 2026-02-12T18:21:22.248 INFO:tasks.workunit.client.0.trial116.stdout: "snap_trimq": "[]", 2026-02-12T18:21:22.249 INFO:tasks.workunit.client.0.trial116.stdout: "snap_trimq_len": 0, 2026-02-12T18:21:22.249 INFO:tasks.workunit.client.0.trial116.stdout: "state": "active+recovery_unfound+degraded", 2026-02-12T18:21:22.249 INFO:tasks.workunit.client.0.trial116.stdout: "epoch": 53, 2026-02-12T18:21:22.249 INFO:tasks.workunit.client.0.trial116.stdout: "up": [ 2026-02-12T18:21:22.249 INFO:tasks.workunit.client.0.trial116.stdout: 3, 2026-02-12T18:21:22.249 INFO:tasks.workunit.client.0.trial116.stdout: 1, 2026-02-12T18:21:22.249 INFO:tasks.workunit.client.0.trial116.stdout: 4, 2026-02-12T18:21:22.249 INFO:tasks.workunit.client.0.trial116.stdout: 0, 2026-02-12T18:21:22.249 INFO:tasks.workunit.client.0.trial116.stdout: 2 2026-02-12T18:21:22.249 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:21:22.249 INFO:tasks.workunit.client.0.trial116.stdout: "acting": [ 2026-02-12T18:21:22.249 INFO:tasks.workunit.client.0.trial116.stdout: 3, 2026-02-12T18:21:22.250 INFO:tasks.workunit.client.0.trial116.stdout: 1, 2026-02-12T18:21:22.250 INFO:tasks.workunit.client.0.trial116.stdout: 4, 2026-02-12T18:21:22.250 INFO:tasks.workunit.client.0.trial116.stdout: 0, 2026-02-12T18:21:22.250 INFO:tasks.workunit.client.0.trial116.stdout: 2 2026-02-12T18:21:22.250 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:21:22.250 INFO:tasks.workunit.client.0.trial116.stdout: "acting_recovery_backfill": [ 2026-02-12T18:21:22.250 INFO:tasks.workunit.client.0.trial116.stdout: "0(3)", 2026-02-12T18:21:22.250 INFO:tasks.workunit.client.0.trial116.stdout: "1(1)", 2026-02-12T18:21:22.250 INFO:tasks.workunit.client.0.trial116.stdout: "2(4)", 2026-02-12T18:21:22.250 INFO:tasks.workunit.client.0.trial116.stdout: "3(0)", 2026-02-12T18:21:22.250 INFO:tasks.workunit.client.0.trial116.stdout: "4(2)" 2026-02-12T18:21:22.250 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:21:22.250 INFO:tasks.workunit.client.0.trial116.stdout: "info": { 2026-02-12T18:21:22.251 INFO:tasks.workunit.client.0.trial116.stdout: "pgid": "2.0s0", 2026-02-12T18:21:22.251 INFO:tasks.workunit.client.0.trial116.stdout: "shard": "0", 2026-02-12T18:21:22.251 INFO:tasks.workunit.client.0.trial116.stdout: "last_update": "46'101", 2026-02-12T18:21:22.251 INFO:tasks.workunit.client.0.trial116.stdout: "last_complete": "46'75", 2026-02-12T18:21:22.251 INFO:tasks.workunit.client.0.trial116.stdout: "log_tail": "0'0", 2026-02-12T18:21:22.251 INFO:tasks.workunit.client.0.trial116.stdout: "last_user_version": 101, 2026-02-12T18:21:22.251 INFO:tasks.workunit.client.0.trial116.stdout: "last_backfill": "MAX", 2026-02-12T18:21:22.251 INFO:tasks.workunit.client.0.trial116.stdout: "partial_writes_last_complete": [], 2026-02-12T18:21:22.251 INFO:tasks.workunit.client.0.trial116.stdout: "partial_writes_last_complete_epoch": "52", 2026-02-12T18:21:22.251 INFO:tasks.workunit.client.0.trial116.stdout: "purged_snaps": [], 2026-02-12T18:21:22.251 INFO:tasks.workunit.client.0.trial116.stdout: "history": { 2026-02-12T18:21:22.251 INFO:tasks.workunit.client.0.trial116.stdout: "epoch_created": 40, 2026-02-12T18:21:22.251 INFO:tasks.workunit.client.0.trial116.stdout: "epoch_pool_created": 40, 2026-02-12T18:21:22.252 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_started": 53, 2026-02-12T18:21:22.252 INFO:tasks.workunit.client.0.trial116.stdout: "last_interval_started": 52, 2026-02-12T18:21:22.252 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_clean": 41, 2026-02-12T18:21:22.252 INFO:tasks.workunit.client.0.trial116.stdout: "last_interval_clean": 40, 2026-02-12T18:21:22.252 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_split": 0, 2026-02-12T18:21:22.252 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_marked_full": 0, 2026-02-12T18:21:22.252 INFO:tasks.workunit.client.0.trial116.stdout: "same_up_since": 52, 2026-02-12T18:21:22.252 INFO:tasks.workunit.client.0.trial116.stdout: "same_interval_since": 52, 2026-02-12T18:21:22.252 INFO:tasks.workunit.client.0.trial116.stdout: "same_primary_since": 40, 2026-02-12T18:21:22.252 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub": "0'0", 2026-02-12T18:21:22.252 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub_stamp": "2026-02-12T18:20:55.921870+0000", 2026-02-12T18:21:22.252 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub": "0'0", 2026-02-12T18:21:22.252 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub_stamp": "2026-02-12T18:20:55.921870+0000", 2026-02-12T18:21:22.253 INFO:tasks.workunit.client.0.trial116.stdout: "last_clean_scrub_stamp": "2026-02-12T18:20:55.921870+0000", 2026-02-12T18:21:22.253 INFO:tasks.workunit.client.0.trial116.stdout: "prior_readable_until_ub": 0 2026-02-12T18:21:22.253 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:21:22.253 INFO:tasks.workunit.client.0.trial116.stdout: "stats": { 2026-02-12T18:21:22.253 INFO:tasks.workunit.client.0.trial116.stdout: "version": "46'101", 2026-02-12T18:21:22.253 INFO:tasks.workunit.client.0.trial116.stdout: "reported_seq": 363, 2026-02-12T18:21:22.253 INFO:tasks.workunit.client.0.trial116.stdout: "reported_epoch": 53, 2026-02-12T18:21:22.253 INFO:tasks.workunit.client.0.trial116.stdout: "state": "active+recovery_unfound+degraded", 2026-02-12T18:21:22.253 INFO:tasks.workunit.client.0.trial116.stdout: "last_fresh": "2026-02-12T18:21:06.703050+0000", 2026-02-12T18:21:22.253 INFO:tasks.workunit.client.0.trial116.stdout: "last_change": "2026-02-12T18:21:06.703050+0000", 2026-02-12T18:21:22.253 INFO:tasks.workunit.client.0.trial116.stdout: "last_active": "2026-02-12T18:21:06.703050+0000", 2026-02-12T18:21:22.253 INFO:tasks.workunit.client.0.trial116.stdout: "last_peered": "2026-02-12T18:21:06.703050+0000", 2026-02-12T18:21:22.254 INFO:tasks.workunit.client.0.trial116.stdout: "last_clean": "2026-02-12T18:21:01.121322+0000", 2026-02-12T18:21:22.254 INFO:tasks.workunit.client.0.trial116.stdout: "last_became_active": "2026-02-12T18:21:06.701487+0000", 2026-02-12T18:21:22.254 INFO:tasks.workunit.client.0.trial116.stdout: "last_became_peered": "2026-02-12T18:21:06.701487+0000", 2026-02-12T18:21:22.254 INFO:tasks.workunit.client.0.trial116.stdout: "last_unstale": "2026-02-12T18:21:06.703050+0000", 2026-02-12T18:21:22.254 INFO:tasks.workunit.client.0.trial116.stdout: "last_undegraded": "2026-02-12T18:21:06.700149+0000", 2026-02-12T18:21:22.254 INFO:tasks.workunit.client.0.trial116.stdout: "last_fullsized": "2026-02-12T18:21:06.703050+0000", 2026-02-12T18:21:22.254 INFO:tasks.workunit.client.0.trial116.stdout: "mapping_epoch": 52, 2026-02-12T18:21:22.254 INFO:tasks.workunit.client.0.trial116.stdout: "log_start": "0'0", 2026-02-12T18:21:22.254 INFO:tasks.workunit.client.0.trial116.stdout: "ondisk_log_start": "0'0", 2026-02-12T18:21:22.254 INFO:tasks.workunit.client.0.trial116.stdout: "created": 40, 2026-02-12T18:21:22.254 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_clean": 41, 2026-02-12T18:21:22.254 INFO:tasks.workunit.client.0.trial116.stdout: "parent": "0.0", 2026-02-12T18:21:22.254 INFO:tasks.workunit.client.0.trial116.stdout: "parent_split_bits": 0, 2026-02-12T18:21:22.254 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub": "0'0", 2026-02-12T18:21:22.255 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub_stamp": "2026-02-12T18:20:55.921870+0000", 2026-02-12T18:21:22.255 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub": "0'0", 2026-02-12T18:21:22.255 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub_stamp": "2026-02-12T18:20:55.921870+0000", 2026-02-12T18:21:22.255 INFO:tasks.workunit.client.0.trial116.stdout: "last_clean_scrub_stamp": "2026-02-12T18:20:55.921870+0000", 2026-02-12T18:21:22.255 INFO:tasks.workunit.client.0.trial116.stdout: "objects_scrubbed": 0, 2026-02-12T18:21:22.255 INFO:tasks.workunit.client.0.trial116.stdout: "log_size": 101, 2026-02-12T18:21:22.255 INFO:tasks.workunit.client.0.trial116.stdout: "log_dups_size": 0, 2026-02-12T18:21:22.255 INFO:tasks.workunit.client.0.trial116.stdout: "ondisk_log_size": 101, 2026-02-12T18:21:22.255 INFO:tasks.workunit.client.0.trial116.stdout: "stats_invalid": false, 2026-02-12T18:21:22.255 INFO:tasks.workunit.client.0.trial116.stdout: "dirty_stats_invalid": false, 2026-02-12T18:21:22.255 INFO:tasks.workunit.client.0.trial116.stdout: "omap_stats_invalid": false, 2026-02-12T18:21:22.255 INFO:tasks.workunit.client.0.trial116.stdout: "hitset_stats_invalid": false, 2026-02-12T18:21:22.255 INFO:tasks.workunit.client.0.trial116.stdout: "hitset_bytes_stats_invalid": false, 2026-02-12T18:21:22.256 INFO:tasks.workunit.client.0.trial116.stdout: "pin_stats_invalid": false, 2026-02-12T18:21:22.256 INFO:tasks.workunit.client.0.trial116.stdout: "manifest_stats_invalid": false, 2026-02-12T18:21:22.256 INFO:tasks.workunit.client.0.trial116.stdout: "snaptrimq_len": 0, 2026-02-12T18:21:22.256 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub_duration": 0, 2026-02-12T18:21:22.256 INFO:tasks.workunit.client.0.trial116.stdout: "scrub_schedule": "no scrub is scheduled", 2026-02-12T18:21:22.256 INFO:tasks.workunit.client.0.trial116.stdout: "scrub_duration": 0, 2026-02-12T18:21:22.256 INFO:tasks.workunit.client.0.trial116.stdout: "objects_trimmed": 0, 2026-02-12T18:21:22.256 INFO:tasks.workunit.client.0.trial116.stdout: "snaptrim_duration": 0, 2026-02-12T18:21:22.256 INFO:tasks.workunit.client.0.trial116.stdout: "stat_sum": { 2026-02-12T18:21:22.256 INFO:tasks.workunit.client.0.trial116.stdout: "num_bytes": 413696, 2026-02-12T18:21:22.256 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects": 101, 2026-02-12T18:21:22.256 INFO:tasks.workunit.client.0.trial116.stdout: "num_object_clones": 0, 2026-02-12T18:21:22.256 INFO:tasks.workunit.client.0.trial116.stdout: "num_object_copies": 505, 2026-02-12T18:21:22.257 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_missing_on_primary": 1, 2026-02-12T18:21:22.257 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_missing": 1, 2026-02-12T18:21:22.257 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_degraded": 3, 2026-02-12T18:21:22.257 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_misplaced": 0, 2026-02-12T18:21:22.257 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_unfound": 1, 2026-02-12T18:21:22.257 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_dirty": 101, 2026-02-12T18:21:22.257 INFO:tasks.workunit.client.0.trial116.stdout: "num_whiteouts": 0, 2026-02-12T18:21:22.257 INFO:tasks.workunit.client.0.trial116.stdout: "num_read": 0, 2026-02-12T18:21:22.257 INFO:tasks.workunit.client.0.trial116.stdout: "num_read_kb": 0, 2026-02-12T18:21:22.257 INFO:tasks.workunit.client.0.trial116.stdout: "num_write": 101, 2026-02-12T18:21:22.257 INFO:tasks.workunit.client.0.trial116.stdout: "num_write_kb": 404, 2026-02-12T18:21:22.257 INFO:tasks.workunit.client.0.trial116.stdout: "num_scrub_errors": 0, 2026-02-12T18:21:22.257 INFO:tasks.workunit.client.0.trial116.stdout: "num_shallow_scrub_errors": 0, 2026-02-12T18:21:22.258 INFO:tasks.workunit.client.0.trial116.stdout: "num_deep_scrub_errors": 0, 2026-02-12T18:21:22.258 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_recovered": 0, 2026-02-12T18:21:22.258 INFO:tasks.workunit.client.0.trial116.stdout: "num_bytes_recovered": 0, 2026-02-12T18:21:22.258 INFO:tasks.workunit.client.0.trial116.stdout: "num_keys_recovered": 0, 2026-02-12T18:21:22.258 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_omap": 0, 2026-02-12T18:21:22.258 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_hit_set_archive": 0, 2026-02-12T18:21:22.258 INFO:tasks.workunit.client.0.trial116.stdout: "num_bytes_hit_set_archive": 0, 2026-02-12T18:21:22.258 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush": 0, 2026-02-12T18:21:22.258 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush_kb": 0, 2026-02-12T18:21:22.258 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict": 0, 2026-02-12T18:21:22.258 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict_kb": 0, 2026-02-12T18:21:22.258 INFO:tasks.workunit.client.0.trial116.stdout: "num_promote": 0, 2026-02-12T18:21:22.258 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush_mode_high": 0, 2026-02-12T18:21:22.259 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush_mode_low": 0, 2026-02-12T18:21:22.259 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict_mode_some": 0, 2026-02-12T18:21:22.259 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict_mode_full": 0, 2026-02-12T18:21:22.259 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_pinned": 0, 2026-02-12T18:21:22.259 INFO:tasks.workunit.client.0.trial116.stdout: "num_legacy_snapsets": 0, 2026-02-12T18:21:22.259 INFO:tasks.workunit.client.0.trial116.stdout: "num_large_omap_objects": 0, 2026-02-12T18:21:22.259 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_manifest": 0, 2026-02-12T18:21:22.259 INFO:tasks.workunit.client.0.trial116.stdout: "num_omap_bytes": 0, 2026-02-12T18:21:22.259 INFO:tasks.workunit.client.0.trial116.stdout: "num_omap_keys": 0, 2026-02-12T18:21:22.259 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_repaired": 0 2026-02-12T18:21:22.259 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:21:22.259 INFO:tasks.workunit.client.0.trial116.stdout: "up": [ 2026-02-12T18:21:22.259 INFO:tasks.workunit.client.0.trial116.stdout: 3, 2026-02-12T18:21:22.259 INFO:tasks.workunit.client.0.trial116.stdout: 1, 2026-02-12T18:21:22.260 INFO:tasks.workunit.client.0.trial116.stdout: 4, 2026-02-12T18:21:22.260 INFO:tasks.workunit.client.0.trial116.stdout: 0, 2026-02-12T18:21:22.260 INFO:tasks.workunit.client.0.trial116.stdout: 2 2026-02-12T18:21:22.260 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:21:22.260 INFO:tasks.workunit.client.0.trial116.stdout: "acting": [ 2026-02-12T18:21:22.260 INFO:tasks.workunit.client.0.trial116.stdout: 3, 2026-02-12T18:21:22.260 INFO:tasks.workunit.client.0.trial116.stdout: 1, 2026-02-12T18:21:22.260 INFO:tasks.workunit.client.0.trial116.stdout: 4, 2026-02-12T18:21:22.260 INFO:tasks.workunit.client.0.trial116.stdout: 0, 2026-02-12T18:21:22.260 INFO:tasks.workunit.client.0.trial116.stdout: 2 2026-02-12T18:21:22.260 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:21:22.260 INFO:tasks.workunit.client.0.trial116.stdout: "avail_no_missing": [ 2026-02-12T18:21:22.260 INFO:tasks.workunit.client.0.trial116.stdout: "0(3)", 2026-02-12T18:21:22.261 INFO:tasks.workunit.client.0.trial116.stdout: "4(2)" 2026-02-12T18:21:22.261 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:21:22.261 INFO:tasks.workunit.client.0.trial116.stdout: "object_location_counts": [ 2026-02-12T18:21:22.261 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:21:22.261 INFO:tasks.workunit.client.0.trial116.stdout: "shards": "0(3),1(1),2(4),3(0),4(2)", 2026-02-12T18:21:22.261 INFO:tasks.workunit.client.0.trial116.stdout: "objects": 100 2026-02-12T18:21:22.261 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:21:22.261 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:21:22.261 INFO:tasks.workunit.client.0.trial116.stdout: "shards": "0(3),4(2)", 2026-02-12T18:21:22.261 INFO:tasks.workunit.client.0.trial116.stdout: "objects": 1 2026-02-12T18:21:22.261 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:21:22.261 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:21:22.261 INFO:tasks.workunit.client.0.trial116.stdout: "blocked_by": [], 2026-02-12T18:21:22.262 INFO:tasks.workunit.client.0.trial116.stdout: "up_primary": 3, 2026-02-12T18:21:22.262 INFO:tasks.workunit.client.0.trial116.stdout: "acting_primary": 3, 2026-02-12T18:21:22.262 INFO:tasks.workunit.client.0.trial116.stdout: "purged_snaps": [] 2026-02-12T18:21:22.262 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:21:22.262 INFO:tasks.workunit.client.0.trial116.stdout: "empty": 0, 2026-02-12T18:21:22.262 INFO:tasks.workunit.client.0.trial116.stdout: "dne": 0, 2026-02-12T18:21:22.262 INFO:tasks.workunit.client.0.trial116.stdout: "incomplete": 0, 2026-02-12T18:21:22.262 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_started": 53, 2026-02-12T18:21:22.262 INFO:tasks.workunit.client.0.trial116.stdout: "hit_set_history": { 2026-02-12T18:21:22.262 INFO:tasks.workunit.client.0.trial116.stdout: "current_last_update": "0'0", 2026-02-12T18:21:22.262 INFO:tasks.workunit.client.0.trial116.stdout: "history": [] 2026-02-12T18:21:22.262 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:21:22.262 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:21:22.263 INFO:tasks.workunit.client.0.trial116.stdout: "peer_info": [ 2026-02-12T18:21:22.263 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:21:22.263 INFO:tasks.workunit.client.0.trial116.stdout: "peer": "0(3)", 2026-02-12T18:21:22.263 INFO:tasks.workunit.client.0.trial116.stdout: "pgid": "2.0s3", 2026-02-12T18:21:22.263 INFO:tasks.workunit.client.0.trial116.stdout: "shard": "3", 2026-02-12T18:21:22.263 INFO:tasks.workunit.client.0.trial116.stdout: "last_update": "46'101", 2026-02-12T18:21:22.263 INFO:tasks.workunit.client.0.trial116.stdout: "last_complete": "46'101", 2026-02-12T18:21:22.263 INFO:tasks.workunit.client.0.trial116.stdout: "log_tail": "0'0", 2026-02-12T18:21:22.263 INFO:tasks.workunit.client.0.trial116.stdout: "last_user_version": 101, 2026-02-12T18:21:22.263 INFO:tasks.workunit.client.0.trial116.stdout: "last_backfill": "MAX", 2026-02-12T18:21:22.263 INFO:tasks.workunit.client.0.trial116.stdout: "partial_writes_last_complete": [], 2026-02-12T18:21:22.263 INFO:tasks.workunit.client.0.trial116.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-12T18:21:22.263 INFO:tasks.workunit.client.0.trial116.stdout: "purged_snaps": [], 2026-02-12T18:21:22.264 INFO:tasks.workunit.client.0.trial116.stdout: "history": { 2026-02-12T18:21:22.264 INFO:tasks.workunit.client.0.trial116.stdout: "epoch_created": 40, 2026-02-12T18:21:22.264 INFO:tasks.workunit.client.0.trial116.stdout: "epoch_pool_created": 40, 2026-02-12T18:21:22.264 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_started": 53, 2026-02-12T18:21:22.264 INFO:tasks.workunit.client.0.trial116.stdout: "last_interval_started": 52, 2026-02-12T18:21:22.264 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_clean": 41, 2026-02-12T18:21:22.264 INFO:tasks.workunit.client.0.trial116.stdout: "last_interval_clean": 40, 2026-02-12T18:21:22.264 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_split": 0, 2026-02-12T18:21:22.264 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_marked_full": 0, 2026-02-12T18:21:22.264 INFO:tasks.workunit.client.0.trial116.stdout: "same_up_since": 52, 2026-02-12T18:21:22.264 INFO:tasks.workunit.client.0.trial116.stdout: "same_interval_since": 52, 2026-02-12T18:21:22.264 INFO:tasks.workunit.client.0.trial116.stdout: "same_primary_since": 40, 2026-02-12T18:21:22.264 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub": "0'0", 2026-02-12T18:21:22.264 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub_stamp": "2026-02-12T18:20:55.921870+0000", 2026-02-12T18:21:22.265 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub": "0'0", 2026-02-12T18:21:22.265 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub_stamp": "2026-02-12T18:20:55.921870+0000", 2026-02-12T18:21:22.265 INFO:tasks.workunit.client.0.trial116.stdout: "last_clean_scrub_stamp": "2026-02-12T18:20:55.921870+0000", 2026-02-12T18:21:22.265 INFO:tasks.workunit.client.0.trial116.stdout: "prior_readable_until_ub": 0 2026-02-12T18:21:22.265 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:21:22.265 INFO:tasks.workunit.client.0.trial116.stdout: "stats": { 2026-02-12T18:21:22.265 INFO:tasks.workunit.client.0.trial116.stdout: "version": "46'101", 2026-02-12T18:21:22.265 INFO:tasks.workunit.client.0.trial116.stdout: "reported_seq": 351, 2026-02-12T18:21:22.265 INFO:tasks.workunit.client.0.trial116.stdout: "reported_epoch": 52, 2026-02-12T18:21:22.265 INFO:tasks.workunit.client.0.trial116.stdout: "state": "active+remapped", 2026-02-12T18:21:22.265 INFO:tasks.workunit.client.0.trial116.stdout: "last_fresh": "2026-02-12T18:21:04.079695+0000", 2026-02-12T18:21:22.265 INFO:tasks.workunit.client.0.trial116.stdout: "last_change": "2026-02-12T18:21:01.540160+0000", 2026-02-12T18:21:22.265 INFO:tasks.workunit.client.0.trial116.stdout: "last_active": "2026-02-12T18:21:04.079695+0000", 2026-02-12T18:21:22.266 INFO:tasks.workunit.client.0.trial116.stdout: "last_peered": "2026-02-12T18:21:04.079695+0000", 2026-02-12T18:21:22.266 INFO:tasks.workunit.client.0.trial116.stdout: "last_clean": "2026-02-12T18:21:01.121322+0000", 2026-02-12T18:21:22.266 INFO:tasks.workunit.client.0.trial116.stdout: "last_became_active": "2026-02-12T18:21:01.540160+0000", 2026-02-12T18:21:22.266 INFO:tasks.workunit.client.0.trial116.stdout: "last_became_peered": "2026-02-12T18:21:01.540160+0000", 2026-02-12T18:21:22.266 INFO:tasks.workunit.client.0.trial116.stdout: "last_unstale": "2026-02-12T18:21:04.079695+0000", 2026-02-12T18:21:22.266 INFO:tasks.workunit.client.0.trial116.stdout: "last_undegraded": "2026-02-12T18:21:01.539099+0000", 2026-02-12T18:21:22.266 INFO:tasks.workunit.client.0.trial116.stdout: "last_fullsized": "2026-02-12T18:21:01.539009+0000", 2026-02-12T18:21:22.266 INFO:tasks.workunit.client.0.trial116.stdout: "mapping_epoch": 52, 2026-02-12T18:21:22.266 INFO:tasks.workunit.client.0.trial116.stdout: "log_start": "0'0", 2026-02-12T18:21:22.266 INFO:tasks.workunit.client.0.trial116.stdout: "ondisk_log_start": "0'0", 2026-02-12T18:21:22.266 INFO:tasks.workunit.client.0.trial116.stdout: "created": 40, 2026-02-12T18:21:22.266 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_clean": 41, 2026-02-12T18:21:22.267 INFO:tasks.workunit.client.0.trial116.stdout: "parent": "0.0", 2026-02-12T18:21:22.267 INFO:tasks.workunit.client.0.trial116.stdout: "parent_split_bits": 0, 2026-02-12T18:21:22.267 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub": "0'0", 2026-02-12T18:21:22.267 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub_stamp": "2026-02-12T18:20:55.921870+0000", 2026-02-12T18:21:22.267 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub": "0'0", 2026-02-12T18:21:22.267 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub_stamp": "2026-02-12T18:20:55.921870+0000", 2026-02-12T18:21:22.267 INFO:tasks.workunit.client.0.trial116.stdout: "last_clean_scrub_stamp": "2026-02-12T18:20:55.921870+0000", 2026-02-12T18:21:22.267 INFO:tasks.workunit.client.0.trial116.stdout: "objects_scrubbed": 0, 2026-02-12T18:21:22.267 INFO:tasks.workunit.client.0.trial116.stdout: "log_size": 101, 2026-02-12T18:21:22.267 INFO:tasks.workunit.client.0.trial116.stdout: "log_dups_size": 0, 2026-02-12T18:21:22.267 INFO:tasks.workunit.client.0.trial116.stdout: "ondisk_log_size": 101, 2026-02-12T18:21:22.267 INFO:tasks.workunit.client.0.trial116.stdout: "stats_invalid": false, 2026-02-12T18:21:22.267 INFO:tasks.workunit.client.0.trial116.stdout: "dirty_stats_invalid": false, 2026-02-12T18:21:22.268 INFO:tasks.workunit.client.0.trial116.stdout: "omap_stats_invalid": false, 2026-02-12T18:21:22.268 INFO:tasks.workunit.client.0.trial116.stdout: "hitset_stats_invalid": false, 2026-02-12T18:21:22.268 INFO:tasks.workunit.client.0.trial116.stdout: "hitset_bytes_stats_invalid": false, 2026-02-12T18:21:22.268 INFO:tasks.workunit.client.0.trial116.stdout: "pin_stats_invalid": false, 2026-02-12T18:21:22.268 INFO:tasks.workunit.client.0.trial116.stdout: "manifest_stats_invalid": false, 2026-02-12T18:21:22.268 INFO:tasks.workunit.client.0.trial116.stdout: "snaptrimq_len": 0, 2026-02-12T18:21:22.268 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub_duration": 0, 2026-02-12T18:21:22.268 INFO:tasks.workunit.client.0.trial116.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-13T20:08:33.664778+0000", 2026-02-12T18:21:22.268 INFO:tasks.workunit.client.0.trial116.stdout: "scrub_duration": 0, 2026-02-12T18:21:22.268 INFO:tasks.workunit.client.0.trial116.stdout: "objects_trimmed": 0, 2026-02-12T18:21:22.268 INFO:tasks.workunit.client.0.trial116.stdout: "snaptrim_duration": 0, 2026-02-12T18:21:22.268 INFO:tasks.workunit.client.0.trial116.stdout: "stat_sum": { 2026-02-12T18:21:22.268 INFO:tasks.workunit.client.0.trial116.stdout: "num_bytes": 413696, 2026-02-12T18:21:22.269 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects": 101, 2026-02-12T18:21:22.269 INFO:tasks.workunit.client.0.trial116.stdout: "num_object_clones": 0, 2026-02-12T18:21:22.269 INFO:tasks.workunit.client.0.trial116.stdout: "num_object_copies": 505, 2026-02-12T18:21:22.269 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_missing_on_primary": 0, 2026-02-12T18:21:22.269 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_missing": 0, 2026-02-12T18:21:22.269 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_degraded": 101, 2026-02-12T18:21:22.269 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_misplaced": 0, 2026-02-12T18:21:22.269 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_unfound": 0, 2026-02-12T18:21:22.269 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_dirty": 101, 2026-02-12T18:21:22.269 INFO:tasks.workunit.client.0.trial116.stdout: "num_whiteouts": 0, 2026-02-12T18:21:22.269 INFO:tasks.workunit.client.0.trial116.stdout: "num_read": 0, 2026-02-12T18:21:22.269 INFO:tasks.workunit.client.0.trial116.stdout: "num_read_kb": 0, 2026-02-12T18:21:22.269 INFO:tasks.workunit.client.0.trial116.stdout: "num_write": 101, 2026-02-12T18:21:22.270 INFO:tasks.workunit.client.0.trial116.stdout: "num_write_kb": 404, 2026-02-12T18:21:22.270 INFO:tasks.workunit.client.0.trial116.stdout: "num_scrub_errors": 0, 2026-02-12T18:21:22.270 INFO:tasks.workunit.client.0.trial116.stdout: "num_shallow_scrub_errors": 0, 2026-02-12T18:21:22.270 INFO:tasks.workunit.client.0.trial116.stdout: "num_deep_scrub_errors": 0, 2026-02-12T18:21:22.270 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_recovered": 0, 2026-02-12T18:21:22.270 INFO:tasks.workunit.client.0.trial116.stdout: "num_bytes_recovered": 0, 2026-02-12T18:21:22.270 INFO:tasks.workunit.client.0.trial116.stdout: "num_keys_recovered": 0, 2026-02-12T18:21:22.270 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_omap": 0, 2026-02-12T18:21:22.270 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_hit_set_archive": 0, 2026-02-12T18:21:22.270 INFO:tasks.workunit.client.0.trial116.stdout: "num_bytes_hit_set_archive": 0, 2026-02-12T18:21:22.270 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush": 0, 2026-02-12T18:21:22.270 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush_kb": 0, 2026-02-12T18:21:22.270 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict": 0, 2026-02-12T18:21:22.271 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict_kb": 0, 2026-02-12T18:21:22.271 INFO:tasks.workunit.client.0.trial116.stdout: "num_promote": 0, 2026-02-12T18:21:22.271 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush_mode_high": 0, 2026-02-12T18:21:22.271 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush_mode_low": 0, 2026-02-12T18:21:22.271 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict_mode_some": 0, 2026-02-12T18:21:22.271 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict_mode_full": 0, 2026-02-12T18:21:22.271 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_pinned": 0, 2026-02-12T18:21:22.271 INFO:tasks.workunit.client.0.trial116.stdout: "num_legacy_snapsets": 0, 2026-02-12T18:21:22.271 INFO:tasks.workunit.client.0.trial116.stdout: "num_large_omap_objects": 0, 2026-02-12T18:21:22.271 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_manifest": 0, 2026-02-12T18:21:22.271 INFO:tasks.workunit.client.0.trial116.stdout: "num_omap_bytes": 0, 2026-02-12T18:21:22.271 INFO:tasks.workunit.client.0.trial116.stdout: "num_omap_keys": 0, 2026-02-12T18:21:22.271 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_repaired": 0 2026-02-12T18:21:22.272 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:21:22.272 INFO:tasks.workunit.client.0.trial116.stdout: "up": [ 2026-02-12T18:21:22.272 INFO:tasks.workunit.client.0.trial116.stdout: 3, 2026-02-12T18:21:22.272 INFO:tasks.workunit.client.0.trial116.stdout: 1, 2026-02-12T18:21:22.272 INFO:tasks.workunit.client.0.trial116.stdout: 4, 2026-02-12T18:21:22.272 INFO:tasks.workunit.client.0.trial116.stdout: 0, 2026-02-12T18:21:22.272 INFO:tasks.workunit.client.0.trial116.stdout: 2 2026-02-12T18:21:22.272 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:21:22.272 INFO:tasks.workunit.client.0.trial116.stdout: "acting": [ 2026-02-12T18:21:22.272 INFO:tasks.workunit.client.0.trial116.stdout: 3, 2026-02-12T18:21:22.272 INFO:tasks.workunit.client.0.trial116.stdout: 1, 2026-02-12T18:21:22.272 INFO:tasks.workunit.client.0.trial116.stdout: 4, 2026-02-12T18:21:22.272 INFO:tasks.workunit.client.0.trial116.stdout: 0, 2026-02-12T18:21:22.273 INFO:tasks.workunit.client.0.trial116.stdout: 2 2026-02-12T18:21:22.273 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:21:22.273 INFO:tasks.workunit.client.0.trial116.stdout: "avail_no_missing": [ 2026-02-12T18:21:22.273 INFO:tasks.workunit.client.0.trial116.stdout: "3(0)", 2026-02-12T18:21:22.273 INFO:tasks.workunit.client.0.trial116.stdout: "0(3)", 2026-02-12T18:21:22.273 INFO:tasks.workunit.client.0.trial116.stdout: "1(1)", 2026-02-12T18:21:22.273 INFO:tasks.workunit.client.0.trial116.stdout: "4(2)" 2026-02-12T18:21:22.273 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:21:22.273 INFO:tasks.workunit.client.0.trial116.stdout: "object_location_counts": [ 2026-02-12T18:21:22.273 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:21:22.273 INFO:tasks.workunit.client.0.trial116.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-12T18:21:22.273 INFO:tasks.workunit.client.0.trial116.stdout: "objects": 101 2026-02-12T18:21:22.273 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:21:22.274 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:21:22.274 INFO:tasks.workunit.client.0.trial116.stdout: "blocked_by": [], 2026-02-12T18:21:22.274 INFO:tasks.workunit.client.0.trial116.stdout: "up_primary": 3, 2026-02-12T18:21:22.274 INFO:tasks.workunit.client.0.trial116.stdout: "acting_primary": 3, 2026-02-12T18:21:22.274 INFO:tasks.workunit.client.0.trial116.stdout: "purged_snaps": [] 2026-02-12T18:21:22.274 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:21:22.274 INFO:tasks.workunit.client.0.trial116.stdout: "empty": 0, 2026-02-12T18:21:22.274 INFO:tasks.workunit.client.0.trial116.stdout: "dne": 0, 2026-02-12T18:21:22.274 INFO:tasks.workunit.client.0.trial116.stdout: "incomplete": 0, 2026-02-12T18:21:22.274 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_started": 53, 2026-02-12T18:21:22.274 INFO:tasks.workunit.client.0.trial116.stdout: "hit_set_history": { 2026-02-12T18:21:22.274 INFO:tasks.workunit.client.0.trial116.stdout: "current_last_update": "0'0", 2026-02-12T18:21:22.274 INFO:tasks.workunit.client.0.trial116.stdout: "history": [] 2026-02-12T18:21:22.274 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:21:22.275 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:21:22.275 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:21:22.275 INFO:tasks.workunit.client.0.trial116.stdout: "peer": "1(1)", 2026-02-12T18:21:22.275 INFO:tasks.workunit.client.0.trial116.stdout: "pgid": "2.0s1", 2026-02-12T18:21:22.275 INFO:tasks.workunit.client.0.trial116.stdout: "shard": "1", 2026-02-12T18:21:22.275 INFO:tasks.workunit.client.0.trial116.stdout: "last_update": "46'101", 2026-02-12T18:21:22.275 INFO:tasks.workunit.client.0.trial116.stdout: "last_complete": "46'101", 2026-02-12T18:21:22.275 INFO:tasks.workunit.client.0.trial116.stdout: "log_tail": "0'0", 2026-02-12T18:21:22.275 INFO:tasks.workunit.client.0.trial116.stdout: "last_user_version": 101, 2026-02-12T18:21:22.275 INFO:tasks.workunit.client.0.trial116.stdout: "last_backfill": "MAX", 2026-02-12T18:21:22.275 INFO:tasks.workunit.client.0.trial116.stdout: "partial_writes_last_complete": [], 2026-02-12T18:21:22.275 INFO:tasks.workunit.client.0.trial116.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-12T18:21:22.275 INFO:tasks.workunit.client.0.trial116.stdout: "purged_snaps": [], 2026-02-12T18:21:22.276 INFO:tasks.workunit.client.0.trial116.stdout: "history": { 2026-02-12T18:21:22.276 INFO:tasks.workunit.client.0.trial116.stdout: "epoch_created": 40, 2026-02-12T18:21:22.276 INFO:tasks.workunit.client.0.trial116.stdout: "epoch_pool_created": 40, 2026-02-12T18:21:22.276 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_started": 53, 2026-02-12T18:21:22.276 INFO:tasks.workunit.client.0.trial116.stdout: "last_interval_started": 52, 2026-02-12T18:21:22.276 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_clean": 41, 2026-02-12T18:21:22.276 INFO:tasks.workunit.client.0.trial116.stdout: "last_interval_clean": 40, 2026-02-12T18:21:22.276 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_split": 0, 2026-02-12T18:21:22.276 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_marked_full": 0, 2026-02-12T18:21:22.276 INFO:tasks.workunit.client.0.trial116.stdout: "same_up_since": 52, 2026-02-12T18:21:22.276 INFO:tasks.workunit.client.0.trial116.stdout: "same_interval_since": 52, 2026-02-12T18:21:22.276 INFO:tasks.workunit.client.0.trial116.stdout: "same_primary_since": 40, 2026-02-12T18:21:22.276 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub": "0'0", 2026-02-12T18:21:22.277 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub_stamp": "2026-02-12T18:20:55.921870+0000", 2026-02-12T18:21:22.277 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub": "0'0", 2026-02-12T18:21:22.277 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub_stamp": "2026-02-12T18:20:55.921870+0000", 2026-02-12T18:21:22.277 INFO:tasks.workunit.client.0.trial116.stdout: "last_clean_scrub_stamp": "2026-02-12T18:20:55.921870+0000", 2026-02-12T18:21:22.277 INFO:tasks.workunit.client.0.trial116.stdout: "prior_readable_until_ub": 0 2026-02-12T18:21:22.277 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:21:22.277 INFO:tasks.workunit.client.0.trial116.stdout: "stats": { 2026-02-12T18:21:22.277 INFO:tasks.workunit.client.0.trial116.stdout: "version": "46'101", 2026-02-12T18:21:22.277 INFO:tasks.workunit.client.0.trial116.stdout: "reported_seq": 132, 2026-02-12T18:21:22.277 INFO:tasks.workunit.client.0.trial116.stdout: "reported_epoch": 46, 2026-02-12T18:21:22.277 INFO:tasks.workunit.client.0.trial116.stdout: "state": "active+remapped", 2026-02-12T18:21:22.277 INFO:tasks.workunit.client.0.trial116.stdout: "last_fresh": "2026-02-12T18:21:04.079695+0000", 2026-02-12T18:21:22.277 INFO:tasks.workunit.client.0.trial116.stdout: "last_change": "2026-02-12T18:21:01.540160+0000", 2026-02-12T18:21:22.277 INFO:tasks.workunit.client.0.trial116.stdout: "last_active": "2026-02-12T18:21:04.079695+0000", 2026-02-12T18:21:22.278 INFO:tasks.workunit.client.0.trial116.stdout: "last_peered": "2026-02-12T18:21:04.079695+0000", 2026-02-12T18:21:22.278 INFO:tasks.workunit.client.0.trial116.stdout: "last_clean": "2026-02-12T18:21:01.121322+0000", 2026-02-12T18:21:22.278 INFO:tasks.workunit.client.0.trial116.stdout: "last_became_active": "2026-02-12T18:21:01.540160+0000", 2026-02-12T18:21:22.278 INFO:tasks.workunit.client.0.trial116.stdout: "last_became_peered": "2026-02-12T18:21:01.540160+0000", 2026-02-12T18:21:22.278 INFO:tasks.workunit.client.0.trial116.stdout: "last_unstale": "2026-02-12T18:21:04.079695+0000", 2026-02-12T18:21:22.278 INFO:tasks.workunit.client.0.trial116.stdout: "last_undegraded": "2026-02-12T18:21:01.539099+0000", 2026-02-12T18:21:22.278 INFO:tasks.workunit.client.0.trial116.stdout: "last_fullsized": "2026-02-12T18:21:01.539009+0000", 2026-02-12T18:21:22.278 INFO:tasks.workunit.client.0.trial116.stdout: "mapping_epoch": 52, 2026-02-12T18:21:22.278 INFO:tasks.workunit.client.0.trial116.stdout: "log_start": "0'0", 2026-02-12T18:21:22.278 INFO:tasks.workunit.client.0.trial116.stdout: "ondisk_log_start": "0'0", 2026-02-12T18:21:22.278 INFO:tasks.workunit.client.0.trial116.stdout: "created": 40, 2026-02-12T18:21:22.278 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_clean": 41, 2026-02-12T18:21:22.278 INFO:tasks.workunit.client.0.trial116.stdout: "parent": "0.0", 2026-02-12T18:21:22.279 INFO:tasks.workunit.client.0.trial116.stdout: "parent_split_bits": 0, 2026-02-12T18:21:22.279 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub": "0'0", 2026-02-12T18:21:22.279 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub_stamp": "2026-02-12T18:20:55.921870+0000", 2026-02-12T18:21:22.279 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub": "0'0", 2026-02-12T18:21:22.279 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub_stamp": "2026-02-12T18:20:55.921870+0000", 2026-02-12T18:21:22.279 INFO:tasks.workunit.client.0.trial116.stdout: "last_clean_scrub_stamp": "2026-02-12T18:20:55.921870+0000", 2026-02-12T18:21:22.279 INFO:tasks.workunit.client.0.trial116.stdout: "objects_scrubbed": 0, 2026-02-12T18:21:22.279 INFO:tasks.workunit.client.0.trial116.stdout: "log_size": 101, 2026-02-12T18:21:22.279 INFO:tasks.workunit.client.0.trial116.stdout: "log_dups_size": 0, 2026-02-12T18:21:22.279 INFO:tasks.workunit.client.0.trial116.stdout: "ondisk_log_size": 101, 2026-02-12T18:21:22.279 INFO:tasks.workunit.client.0.trial116.stdout: "stats_invalid": false, 2026-02-12T18:21:22.279 INFO:tasks.workunit.client.0.trial116.stdout: "dirty_stats_invalid": false, 2026-02-12T18:21:22.279 INFO:tasks.workunit.client.0.trial116.stdout: "omap_stats_invalid": false, 2026-02-12T18:21:22.279 INFO:tasks.workunit.client.0.trial116.stdout: "hitset_stats_invalid": false, 2026-02-12T18:21:22.280 INFO:tasks.workunit.client.0.trial116.stdout: "hitset_bytes_stats_invalid": false, 2026-02-12T18:21:22.280 INFO:tasks.workunit.client.0.trial116.stdout: "pin_stats_invalid": false, 2026-02-12T18:21:22.280 INFO:tasks.workunit.client.0.trial116.stdout: "manifest_stats_invalid": false, 2026-02-12T18:21:22.280 INFO:tasks.workunit.client.0.trial116.stdout: "snaptrimq_len": 0, 2026-02-12T18:21:22.280 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub_duration": 0, 2026-02-12T18:21:22.280 INFO:tasks.workunit.client.0.trial116.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-13T20:08:33.664778+0000", 2026-02-12T18:21:22.280 INFO:tasks.workunit.client.0.trial116.stdout: "scrub_duration": 0, 2026-02-12T18:21:22.280 INFO:tasks.workunit.client.0.trial116.stdout: "objects_trimmed": 0, 2026-02-12T18:21:22.280 INFO:tasks.workunit.client.0.trial116.stdout: "snaptrim_duration": 0, 2026-02-12T18:21:22.280 INFO:tasks.workunit.client.0.trial116.stdout: "stat_sum": { 2026-02-12T18:21:22.280 INFO:tasks.workunit.client.0.trial116.stdout: "num_bytes": 413696, 2026-02-12T18:21:22.280 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects": 101, 2026-02-12T18:21:22.280 INFO:tasks.workunit.client.0.trial116.stdout: "num_object_clones": 0, 2026-02-12T18:21:22.281 INFO:tasks.workunit.client.0.trial116.stdout: "num_object_copies": 505, 2026-02-12T18:21:22.281 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_missing_on_primary": 0, 2026-02-12T18:21:22.281 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_missing": 1, 2026-02-12T18:21:22.281 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_degraded": 101, 2026-02-12T18:21:22.281 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_misplaced": 0, 2026-02-12T18:21:22.281 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_unfound": 0, 2026-02-12T18:21:22.281 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_dirty": 101, 2026-02-12T18:21:22.281 INFO:tasks.workunit.client.0.trial116.stdout: "num_whiteouts": 0, 2026-02-12T18:21:22.281 INFO:tasks.workunit.client.0.trial116.stdout: "num_read": 0, 2026-02-12T18:21:22.281 INFO:tasks.workunit.client.0.trial116.stdout: "num_read_kb": 0, 2026-02-12T18:21:22.281 INFO:tasks.workunit.client.0.trial116.stdout: "num_write": 101, 2026-02-12T18:21:22.281 INFO:tasks.workunit.client.0.trial116.stdout: "num_write_kb": 404, 2026-02-12T18:21:22.281 INFO:tasks.workunit.client.0.trial116.stdout: "num_scrub_errors": 0, 2026-02-12T18:21:22.282 INFO:tasks.workunit.client.0.trial116.stdout: "num_shallow_scrub_errors": 0, 2026-02-12T18:21:22.282 INFO:tasks.workunit.client.0.trial116.stdout: "num_deep_scrub_errors": 0, 2026-02-12T18:21:22.282 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_recovered": 0, 2026-02-12T18:21:22.282 INFO:tasks.workunit.client.0.trial116.stdout: "num_bytes_recovered": 0, 2026-02-12T18:21:22.282 INFO:tasks.workunit.client.0.trial116.stdout: "num_keys_recovered": 0, 2026-02-12T18:21:22.282 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_omap": 0, 2026-02-12T18:21:22.282 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_hit_set_archive": 0, 2026-02-12T18:21:22.282 INFO:tasks.workunit.client.0.trial116.stdout: "num_bytes_hit_set_archive": 0, 2026-02-12T18:21:22.282 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush": 0, 2026-02-12T18:21:22.282 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush_kb": 0, 2026-02-12T18:21:22.282 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict": 0, 2026-02-12T18:21:22.282 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict_kb": 0, 2026-02-12T18:21:22.282 INFO:tasks.workunit.client.0.trial116.stdout: "num_promote": 0, 2026-02-12T18:21:22.283 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush_mode_high": 0, 2026-02-12T18:21:22.283 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush_mode_low": 0, 2026-02-12T18:21:22.283 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict_mode_some": 0, 2026-02-12T18:21:22.283 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict_mode_full": 0, 2026-02-12T18:21:22.283 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_pinned": 0, 2026-02-12T18:21:22.283 INFO:tasks.workunit.client.0.trial116.stdout: "num_legacy_snapsets": 0, 2026-02-12T18:21:22.283 INFO:tasks.workunit.client.0.trial116.stdout: "num_large_omap_objects": 0, 2026-02-12T18:21:22.283 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_manifest": 0, 2026-02-12T18:21:22.283 INFO:tasks.workunit.client.0.trial116.stdout: "num_omap_bytes": 0, 2026-02-12T18:21:22.283 INFO:tasks.workunit.client.0.trial116.stdout: "num_omap_keys": 0, 2026-02-12T18:21:22.283 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_repaired": 0 2026-02-12T18:21:22.283 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:21:22.283 INFO:tasks.workunit.client.0.trial116.stdout: "up": [ 2026-02-12T18:21:22.284 INFO:tasks.workunit.client.0.trial116.stdout: 3, 2026-02-12T18:21:22.284 INFO:tasks.workunit.client.0.trial116.stdout: 1, 2026-02-12T18:21:22.284 INFO:tasks.workunit.client.0.trial116.stdout: 4, 2026-02-12T18:21:22.284 INFO:tasks.workunit.client.0.trial116.stdout: 0, 2026-02-12T18:21:22.284 INFO:tasks.workunit.client.0.trial116.stdout: 2 2026-02-12T18:21:22.284 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:21:22.284 INFO:tasks.workunit.client.0.trial116.stdout: "acting": [ 2026-02-12T18:21:22.284 INFO:tasks.workunit.client.0.trial116.stdout: 3, 2026-02-12T18:21:22.284 INFO:tasks.workunit.client.0.trial116.stdout: 1, 2026-02-12T18:21:22.284 INFO:tasks.workunit.client.0.trial116.stdout: 4, 2026-02-12T18:21:22.284 INFO:tasks.workunit.client.0.trial116.stdout: 0, 2026-02-12T18:21:22.284 INFO:tasks.workunit.client.0.trial116.stdout: 2 2026-02-12T18:21:22.284 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:21:22.284 INFO:tasks.workunit.client.0.trial116.stdout: "avail_no_missing": [ 2026-02-12T18:21:22.285 INFO:tasks.workunit.client.0.trial116.stdout: "3(0)", 2026-02-12T18:21:22.285 INFO:tasks.workunit.client.0.trial116.stdout: "0(3)", 2026-02-12T18:21:22.285 INFO:tasks.workunit.client.0.trial116.stdout: "1(1)", 2026-02-12T18:21:22.285 INFO:tasks.workunit.client.0.trial116.stdout: "4(2)" 2026-02-12T18:21:22.285 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:21:22.285 INFO:tasks.workunit.client.0.trial116.stdout: "object_location_counts": [ 2026-02-12T18:21:22.285 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:21:22.285 INFO:tasks.workunit.client.0.trial116.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-12T18:21:22.285 INFO:tasks.workunit.client.0.trial116.stdout: "objects": 101 2026-02-12T18:21:22.285 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:21:22.285 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:21:22.285 INFO:tasks.workunit.client.0.trial116.stdout: "blocked_by": [], 2026-02-12T18:21:22.286 INFO:tasks.workunit.client.0.trial116.stdout: "up_primary": 3, 2026-02-12T18:21:22.286 INFO:tasks.workunit.client.0.trial116.stdout: "acting_primary": 3, 2026-02-12T18:21:22.286 INFO:tasks.workunit.client.0.trial116.stdout: "purged_snaps": [] 2026-02-12T18:21:22.286 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:21:22.286 INFO:tasks.workunit.client.0.trial116.stdout: "empty": 0, 2026-02-12T18:21:22.286 INFO:tasks.workunit.client.0.trial116.stdout: "dne": 0, 2026-02-12T18:21:22.286 INFO:tasks.workunit.client.0.trial116.stdout: "incomplete": 0, 2026-02-12T18:21:22.286 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_started": 53, 2026-02-12T18:21:22.286 INFO:tasks.workunit.client.0.trial116.stdout: "hit_set_history": { 2026-02-12T18:21:22.286 INFO:tasks.workunit.client.0.trial116.stdout: "current_last_update": "0'0", 2026-02-12T18:21:22.286 INFO:tasks.workunit.client.0.trial116.stdout: "history": [] 2026-02-12T18:21:22.286 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:21:22.286 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:21:22.286 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:21:22.287 INFO:tasks.workunit.client.0.trial116.stdout: "peer": "2(4)", 2026-02-12T18:21:22.287 INFO:tasks.workunit.client.0.trial116.stdout: "pgid": "2.0s4", 2026-02-12T18:21:22.287 INFO:tasks.workunit.client.0.trial116.stdout: "shard": "4", 2026-02-12T18:21:22.287 INFO:tasks.workunit.client.0.trial116.stdout: "last_update": "46'101", 2026-02-12T18:21:22.287 INFO:tasks.workunit.client.0.trial116.stdout: "last_complete": "46'75", 2026-02-12T18:21:22.287 INFO:tasks.workunit.client.0.trial116.stdout: "log_tail": "0'0", 2026-02-12T18:21:22.287 INFO:tasks.workunit.client.0.trial116.stdout: "last_user_version": 101, 2026-02-12T18:21:22.287 INFO:tasks.workunit.client.0.trial116.stdout: "last_backfill": "MAX", 2026-02-12T18:21:22.287 INFO:tasks.workunit.client.0.trial116.stdout: "partial_writes_last_complete": [], 2026-02-12T18:21:22.287 INFO:tasks.workunit.client.0.trial116.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-12T18:21:22.287 INFO:tasks.workunit.client.0.trial116.stdout: "purged_snaps": [], 2026-02-12T18:21:22.287 INFO:tasks.workunit.client.0.trial116.stdout: "history": { 2026-02-12T18:21:22.287 INFO:tasks.workunit.client.0.trial116.stdout: "epoch_created": 40, 2026-02-12T18:21:22.287 INFO:tasks.workunit.client.0.trial116.stdout: "epoch_pool_created": 40, 2026-02-12T18:21:22.287 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_started": 53, 2026-02-12T18:21:22.287 INFO:tasks.workunit.client.0.trial116.stdout: "last_interval_started": 52, 2026-02-12T18:21:22.288 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_clean": 41, 2026-02-12T18:21:22.288 INFO:tasks.workunit.client.0.trial116.stdout: "last_interval_clean": 40, 2026-02-12T18:21:22.288 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_split": 0, 2026-02-12T18:21:22.288 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_marked_full": 0, 2026-02-12T18:21:22.288 INFO:tasks.workunit.client.0.trial116.stdout: "same_up_since": 52, 2026-02-12T18:21:22.288 INFO:tasks.workunit.client.0.trial116.stdout: "same_interval_since": 52, 2026-02-12T18:21:22.288 INFO:tasks.workunit.client.0.trial116.stdout: "same_primary_since": 40, 2026-02-12T18:21:22.288 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub": "0'0", 2026-02-12T18:21:22.288 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub_stamp": "2026-02-12T18:20:55.921870+0000", 2026-02-12T18:21:22.288 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub": "0'0", 2026-02-12T18:21:22.288 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub_stamp": "2026-02-12T18:20:55.921870+0000", 2026-02-12T18:21:22.288 INFO:tasks.workunit.client.0.trial116.stdout: "last_clean_scrub_stamp": "2026-02-12T18:20:55.921870+0000", 2026-02-12T18:21:22.288 INFO:tasks.workunit.client.0.trial116.stdout: "prior_readable_until_ub": 0 2026-02-12T18:21:22.288 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:21:22.288 INFO:tasks.workunit.client.0.trial116.stdout: "stats": { 2026-02-12T18:21:22.289 INFO:tasks.workunit.client.0.trial116.stdout: "version": "46'101", 2026-02-12T18:21:22.289 INFO:tasks.workunit.client.0.trial116.stdout: "reported_seq": 139, 2026-02-12T18:21:22.289 INFO:tasks.workunit.client.0.trial116.stdout: "reported_epoch": 50, 2026-02-12T18:21:22.289 INFO:tasks.workunit.client.0.trial116.stdout: "state": "active+remapped", 2026-02-12T18:21:22.289 INFO:tasks.workunit.client.0.trial116.stdout: "last_fresh": "2026-02-12T18:21:05.658632+0000", 2026-02-12T18:21:22.289 INFO:tasks.workunit.client.0.trial116.stdout: "last_change": "2026-02-12T18:21:05.557578+0000", 2026-02-12T18:21:22.289 INFO:tasks.workunit.client.0.trial116.stdout: "last_active": "2026-02-12T18:21:05.557578+0000", 2026-02-12T18:21:22.289 INFO:tasks.workunit.client.0.trial116.stdout: "last_peered": "2026-02-12T18:21:05.535735+0000", 2026-02-12T18:21:22.289 INFO:tasks.workunit.client.0.trial116.stdout: "last_clean": "2026-02-12T18:21:01.121322+0000", 2026-02-12T18:21:22.289 INFO:tasks.workunit.client.0.trial116.stdout: "last_became_active": "2026-02-12T18:21:01.540160+0000", 2026-02-12T18:21:22.289 INFO:tasks.workunit.client.0.trial116.stdout: "last_became_peered": "2026-02-12T18:21:01.540160+0000", 2026-02-12T18:21:22.289 INFO:tasks.workunit.client.0.trial116.stdout: "last_unstale": "2026-02-12T18:21:05.658632+0000", 2026-02-12T18:21:22.289 INFO:tasks.workunit.client.0.trial116.stdout: "last_undegraded": "2026-02-12T18:21:05.658632+0000", 2026-02-12T18:21:22.289 INFO:tasks.workunit.client.0.trial116.stdout: "last_fullsized": "2026-02-12T18:21:05.658632+0000", 2026-02-12T18:21:22.289 INFO:tasks.workunit.client.0.trial116.stdout: "mapping_epoch": 52, 2026-02-12T18:21:22.290 INFO:tasks.workunit.client.0.trial116.stdout: "log_start": "0'0", 2026-02-12T18:21:22.290 INFO:tasks.workunit.client.0.trial116.stdout: "ondisk_log_start": "0'0", 2026-02-12T18:21:22.290 INFO:tasks.workunit.client.0.trial116.stdout: "created": 40, 2026-02-12T18:21:22.290 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_clean": 41, 2026-02-12T18:21:22.290 INFO:tasks.workunit.client.0.trial116.stdout: "parent": "0.0", 2026-02-12T18:21:22.290 INFO:tasks.workunit.client.0.trial116.stdout: "parent_split_bits": 0, 2026-02-12T18:21:22.290 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub": "0'0", 2026-02-12T18:21:22.290 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub_stamp": "2026-02-12T18:20:55.921870+0000", 2026-02-12T18:21:22.290 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub": "0'0", 2026-02-12T18:21:22.290 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub_stamp": "2026-02-12T18:20:55.921870+0000", 2026-02-12T18:21:22.290 INFO:tasks.workunit.client.0.trial116.stdout: "last_clean_scrub_stamp": "2026-02-12T18:20:55.921870+0000", 2026-02-12T18:21:22.290 INFO:tasks.workunit.client.0.trial116.stdout: "objects_scrubbed": 0, 2026-02-12T18:21:22.290 INFO:tasks.workunit.client.0.trial116.stdout: "log_size": 101, 2026-02-12T18:21:22.290 INFO:tasks.workunit.client.0.trial116.stdout: "log_dups_size": 0, 2026-02-12T18:21:22.290 INFO:tasks.workunit.client.0.trial116.stdout: "ondisk_log_size": 101, 2026-02-12T18:21:22.290 INFO:tasks.workunit.client.0.trial116.stdout: "stats_invalid": false, 2026-02-12T18:21:22.291 INFO:tasks.workunit.client.0.trial116.stdout: "dirty_stats_invalid": false, 2026-02-12T18:21:22.291 INFO:tasks.workunit.client.0.trial116.stdout: "omap_stats_invalid": false, 2026-02-12T18:21:22.291 INFO:tasks.workunit.client.0.trial116.stdout: "hitset_stats_invalid": false, 2026-02-12T18:21:22.291 INFO:tasks.workunit.client.0.trial116.stdout: "hitset_bytes_stats_invalid": false, 2026-02-12T18:21:22.291 INFO:tasks.workunit.client.0.trial116.stdout: "pin_stats_invalid": false, 2026-02-12T18:21:22.291 INFO:tasks.workunit.client.0.trial116.stdout: "manifest_stats_invalid": false, 2026-02-12T18:21:22.291 INFO:tasks.workunit.client.0.trial116.stdout: "snaptrimq_len": 0, 2026-02-12T18:21:22.291 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub_duration": 0, 2026-02-12T18:21:22.291 INFO:tasks.workunit.client.0.trial116.stdout: "scrub_schedule": "no scrub is scheduled", 2026-02-12T18:21:22.291 INFO:tasks.workunit.client.0.trial116.stdout: "scrub_duration": 0, 2026-02-12T18:21:22.291 INFO:tasks.workunit.client.0.trial116.stdout: "objects_trimmed": 0, 2026-02-12T18:21:22.291 INFO:tasks.workunit.client.0.trial116.stdout: "snaptrim_duration": 0, 2026-02-12T18:21:22.291 INFO:tasks.workunit.client.0.trial116.stdout: "stat_sum": { 2026-02-12T18:21:22.291 INFO:tasks.workunit.client.0.trial116.stdout: "num_bytes": 413696, 2026-02-12T18:21:22.291 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects": 101, 2026-02-12T18:21:22.291 INFO:tasks.workunit.client.0.trial116.stdout: "num_object_clones": 0, 2026-02-12T18:21:22.291 INFO:tasks.workunit.client.0.trial116.stdout: "num_object_copies": 505, 2026-02-12T18:21:22.292 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_missing_on_primary": 0, 2026-02-12T18:21:22.292 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_missing": 1, 2026-02-12T18:21:22.292 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_degraded": 0, 2026-02-12T18:21:22.292 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_misplaced": 0, 2026-02-12T18:21:22.292 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_unfound": 0, 2026-02-12T18:21:22.292 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_dirty": 101, 2026-02-12T18:21:22.292 INFO:tasks.workunit.client.0.trial116.stdout: "num_whiteouts": 0, 2026-02-12T18:21:22.292 INFO:tasks.workunit.client.0.trial116.stdout: "num_read": 0, 2026-02-12T18:21:22.292 INFO:tasks.workunit.client.0.trial116.stdout: "num_read_kb": 0, 2026-02-12T18:21:22.292 INFO:tasks.workunit.client.0.trial116.stdout: "num_write": 101, 2026-02-12T18:21:22.292 INFO:tasks.workunit.client.0.trial116.stdout: "num_write_kb": 404, 2026-02-12T18:21:22.292 INFO:tasks.workunit.client.0.trial116.stdout: "num_scrub_errors": 0, 2026-02-12T18:21:22.292 INFO:tasks.workunit.client.0.trial116.stdout: "num_shallow_scrub_errors": 0, 2026-02-12T18:21:22.292 INFO:tasks.workunit.client.0.trial116.stdout: "num_deep_scrub_errors": 0, 2026-02-12T18:21:22.292 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_recovered": 0, 2026-02-12T18:21:22.292 INFO:tasks.workunit.client.0.trial116.stdout: "num_bytes_recovered": 0, 2026-02-12T18:21:22.292 INFO:tasks.workunit.client.0.trial116.stdout: "num_keys_recovered": 0, 2026-02-12T18:21:22.292 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_omap": 0, 2026-02-12T18:21:22.293 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_hit_set_archive": 0, 2026-02-12T18:21:22.293 INFO:tasks.workunit.client.0.trial116.stdout: "num_bytes_hit_set_archive": 0, 2026-02-12T18:21:22.293 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush": 0, 2026-02-12T18:21:22.293 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush_kb": 0, 2026-02-12T18:21:22.293 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict": 0, 2026-02-12T18:21:22.293 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict_kb": 0, 2026-02-12T18:21:22.293 INFO:tasks.workunit.client.0.trial116.stdout: "num_promote": 0, 2026-02-12T18:21:22.293 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush_mode_high": 0, 2026-02-12T18:21:22.293 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush_mode_low": 0, 2026-02-12T18:21:22.293 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict_mode_some": 0, 2026-02-12T18:21:22.293 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict_mode_full": 0, 2026-02-12T18:21:22.293 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_pinned": 0, 2026-02-12T18:21:22.293 INFO:tasks.workunit.client.0.trial116.stdout: "num_legacy_snapsets": 0, 2026-02-12T18:21:22.293 INFO:tasks.workunit.client.0.trial116.stdout: "num_large_omap_objects": 0, 2026-02-12T18:21:22.293 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_manifest": 0, 2026-02-12T18:21:22.293 INFO:tasks.workunit.client.0.trial116.stdout: "num_omap_bytes": 0, 2026-02-12T18:21:22.293 INFO:tasks.workunit.client.0.trial116.stdout: "num_omap_keys": 0, 2026-02-12T18:21:22.293 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_repaired": 0 2026-02-12T18:21:22.294 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:21:22.294 INFO:tasks.workunit.client.0.trial116.stdout: "up": [ 2026-02-12T18:21:22.294 INFO:tasks.workunit.client.0.trial116.stdout: 3, 2026-02-12T18:21:22.294 INFO:tasks.workunit.client.0.trial116.stdout: 1, 2026-02-12T18:21:22.294 INFO:tasks.workunit.client.0.trial116.stdout: 4, 2026-02-12T18:21:22.294 INFO:tasks.workunit.client.0.trial116.stdout: 0, 2026-02-12T18:21:22.294 INFO:tasks.workunit.client.0.trial116.stdout: 2 2026-02-12T18:21:22.294 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:21:22.294 INFO:tasks.workunit.client.0.trial116.stdout: "acting": [ 2026-02-12T18:21:22.294 INFO:tasks.workunit.client.0.trial116.stdout: 3, 2026-02-12T18:21:22.294 INFO:tasks.workunit.client.0.trial116.stdout: 1, 2026-02-12T18:21:22.294 INFO:tasks.workunit.client.0.trial116.stdout: 4, 2026-02-12T18:21:22.294 INFO:tasks.workunit.client.0.trial116.stdout: 0, 2026-02-12T18:21:22.294 INFO:tasks.workunit.client.0.trial116.stdout: 2 2026-02-12T18:21:22.294 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:21:22.294 INFO:tasks.workunit.client.0.trial116.stdout: "avail_no_missing": [], 2026-02-12T18:21:22.294 INFO:tasks.workunit.client.0.trial116.stdout: "object_location_counts": [], 2026-02-12T18:21:22.294 INFO:tasks.workunit.client.0.trial116.stdout: "blocked_by": [], 2026-02-12T18:21:22.294 INFO:tasks.workunit.client.0.trial116.stdout: "up_primary": 3, 2026-02-12T18:21:22.295 INFO:tasks.workunit.client.0.trial116.stdout: "acting_primary": 3, 2026-02-12T18:21:22.295 INFO:tasks.workunit.client.0.trial116.stdout: "purged_snaps": [] 2026-02-12T18:21:22.295 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:21:22.295 INFO:tasks.workunit.client.0.trial116.stdout: "empty": 0, 2026-02-12T18:21:22.295 INFO:tasks.workunit.client.0.trial116.stdout: "dne": 0, 2026-02-12T18:21:22.295 INFO:tasks.workunit.client.0.trial116.stdout: "incomplete": 0, 2026-02-12T18:21:22.295 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_started": 53, 2026-02-12T18:21:22.295 INFO:tasks.workunit.client.0.trial116.stdout: "hit_set_history": { 2026-02-12T18:21:22.295 INFO:tasks.workunit.client.0.trial116.stdout: "current_last_update": "0'0", 2026-02-12T18:21:22.295 INFO:tasks.workunit.client.0.trial116.stdout: "history": [] 2026-02-12T18:21:22.295 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:21:22.295 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:21:22.295 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:21:22.295 INFO:tasks.workunit.client.0.trial116.stdout: "peer": "4(2)", 2026-02-12T18:21:22.295 INFO:tasks.workunit.client.0.trial116.stdout: "pgid": "2.0s2", 2026-02-12T18:21:22.295 INFO:tasks.workunit.client.0.trial116.stdout: "shard": "2", 2026-02-12T18:21:22.295 INFO:tasks.workunit.client.0.trial116.stdout: "last_update": "46'101", 2026-02-12T18:21:22.295 INFO:tasks.workunit.client.0.trial116.stdout: "last_complete": "46'101", 2026-02-12T18:21:22.296 INFO:tasks.workunit.client.0.trial116.stdout: "log_tail": "0'0", 2026-02-12T18:21:22.296 INFO:tasks.workunit.client.0.trial116.stdout: "last_user_version": 101, 2026-02-12T18:21:22.296 INFO:tasks.workunit.client.0.trial116.stdout: "last_backfill": "MAX", 2026-02-12T18:21:22.296 INFO:tasks.workunit.client.0.trial116.stdout: "partial_writes_last_complete": [], 2026-02-12T18:21:22.296 INFO:tasks.workunit.client.0.trial116.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-12T18:21:22.296 INFO:tasks.workunit.client.0.trial116.stdout: "purged_snaps": [], 2026-02-12T18:21:22.296 INFO:tasks.workunit.client.0.trial116.stdout: "history": { 2026-02-12T18:21:22.296 INFO:tasks.workunit.client.0.trial116.stdout: "epoch_created": 40, 2026-02-12T18:21:22.296 INFO:tasks.workunit.client.0.trial116.stdout: "epoch_pool_created": 40, 2026-02-12T18:21:22.296 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_started": 53, 2026-02-12T18:21:22.296 INFO:tasks.workunit.client.0.trial116.stdout: "last_interval_started": 52, 2026-02-12T18:21:22.296 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_clean": 41, 2026-02-12T18:21:22.296 INFO:tasks.workunit.client.0.trial116.stdout: "last_interval_clean": 40, 2026-02-12T18:21:22.296 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_split": 0, 2026-02-12T18:21:22.296 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_marked_full": 0, 2026-02-12T18:21:22.296 INFO:tasks.workunit.client.0.trial116.stdout: "same_up_since": 52, 2026-02-12T18:21:22.296 INFO:tasks.workunit.client.0.trial116.stdout: "same_interval_since": 52, 2026-02-12T18:21:22.296 INFO:tasks.workunit.client.0.trial116.stdout: "same_primary_since": 40, 2026-02-12T18:21:22.297 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub": "0'0", 2026-02-12T18:21:22.297 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub_stamp": "2026-02-12T18:20:55.921870+0000", 2026-02-12T18:21:22.297 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub": "0'0", 2026-02-12T18:21:22.297 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub_stamp": "2026-02-12T18:20:55.921870+0000", 2026-02-12T18:21:22.297 INFO:tasks.workunit.client.0.trial116.stdout: "last_clean_scrub_stamp": "2026-02-12T18:20:55.921870+0000", 2026-02-12T18:21:22.297 INFO:tasks.workunit.client.0.trial116.stdout: "prior_readable_until_ub": 0 2026-02-12T18:21:22.297 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:21:22.297 INFO:tasks.workunit.client.0.trial116.stdout: "stats": { 2026-02-12T18:21:22.297 INFO:tasks.workunit.client.0.trial116.stdout: "version": "46'101", 2026-02-12T18:21:22.297 INFO:tasks.workunit.client.0.trial116.stdout: "reported_seq": 132, 2026-02-12T18:21:22.297 INFO:tasks.workunit.client.0.trial116.stdout: "reported_epoch": 46, 2026-02-12T18:21:22.297 INFO:tasks.workunit.client.0.trial116.stdout: "state": "active+remapped", 2026-02-12T18:21:22.297 INFO:tasks.workunit.client.0.trial116.stdout: "last_fresh": "2026-02-12T18:21:04.079695+0000", 2026-02-12T18:21:22.297 INFO:tasks.workunit.client.0.trial116.stdout: "last_change": "2026-02-12T18:21:01.540160+0000", 2026-02-12T18:21:22.297 INFO:tasks.workunit.client.0.trial116.stdout: "last_active": "2026-02-12T18:21:04.079695+0000", 2026-02-12T18:21:22.297 INFO:tasks.workunit.client.0.trial116.stdout: "last_peered": "2026-02-12T18:21:04.079695+0000", 2026-02-12T18:21:22.297 INFO:tasks.workunit.client.0.trial116.stdout: "last_clean": "2026-02-12T18:21:01.121322+0000", 2026-02-12T18:21:22.297 INFO:tasks.workunit.client.0.trial116.stdout: "last_became_active": "2026-02-12T18:21:01.540160+0000", 2026-02-12T18:21:22.298 INFO:tasks.workunit.client.0.trial116.stdout: "last_became_peered": "2026-02-12T18:21:01.540160+0000", 2026-02-12T18:21:22.298 INFO:tasks.workunit.client.0.trial116.stdout: "last_unstale": "2026-02-12T18:21:04.079695+0000", 2026-02-12T18:21:22.298 INFO:tasks.workunit.client.0.trial116.stdout: "last_undegraded": "2026-02-12T18:21:01.539099+0000", 2026-02-12T18:21:22.298 INFO:tasks.workunit.client.0.trial116.stdout: "last_fullsized": "2026-02-12T18:21:01.539009+0000", 2026-02-12T18:21:22.298 INFO:tasks.workunit.client.0.trial116.stdout: "mapping_epoch": 52, 2026-02-12T18:21:22.298 INFO:tasks.workunit.client.0.trial116.stdout: "log_start": "0'0", 2026-02-12T18:21:22.298 INFO:tasks.workunit.client.0.trial116.stdout: "ondisk_log_start": "0'0", 2026-02-12T18:21:22.298 INFO:tasks.workunit.client.0.trial116.stdout: "created": 40, 2026-02-12T18:21:22.298 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_clean": 41, 2026-02-12T18:21:22.298 INFO:tasks.workunit.client.0.trial116.stdout: "parent": "0.0", 2026-02-12T18:21:22.298 INFO:tasks.workunit.client.0.trial116.stdout: "parent_split_bits": 0, 2026-02-12T18:21:22.298 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub": "0'0", 2026-02-12T18:21:22.298 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub_stamp": "2026-02-12T18:20:55.921870+0000", 2026-02-12T18:21:22.298 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub": "0'0", 2026-02-12T18:21:22.298 INFO:tasks.workunit.client.0.trial116.stdout: "last_deep_scrub_stamp": "2026-02-12T18:20:55.921870+0000", 2026-02-12T18:21:22.298 INFO:tasks.workunit.client.0.trial116.stdout: "last_clean_scrub_stamp": "2026-02-12T18:20:55.921870+0000", 2026-02-12T18:21:22.298 INFO:tasks.workunit.client.0.trial116.stdout: "objects_scrubbed": 0, 2026-02-12T18:21:22.298 INFO:tasks.workunit.client.0.trial116.stdout: "log_size": 101, 2026-02-12T18:21:22.298 INFO:tasks.workunit.client.0.trial116.stdout: "log_dups_size": 0, 2026-02-12T18:21:22.299 INFO:tasks.workunit.client.0.trial116.stdout: "ondisk_log_size": 101, 2026-02-12T18:21:22.299 INFO:tasks.workunit.client.0.trial116.stdout: "stats_invalid": false, 2026-02-12T18:21:22.299 INFO:tasks.workunit.client.0.trial116.stdout: "dirty_stats_invalid": false, 2026-02-12T18:21:22.299 INFO:tasks.workunit.client.0.trial116.stdout: "omap_stats_invalid": false, 2026-02-12T18:21:22.299 INFO:tasks.workunit.client.0.trial116.stdout: "hitset_stats_invalid": false, 2026-02-12T18:21:22.299 INFO:tasks.workunit.client.0.trial116.stdout: "hitset_bytes_stats_invalid": false, 2026-02-12T18:21:22.299 INFO:tasks.workunit.client.0.trial116.stdout: "pin_stats_invalid": false, 2026-02-12T18:21:22.299 INFO:tasks.workunit.client.0.trial116.stdout: "manifest_stats_invalid": false, 2026-02-12T18:21:22.299 INFO:tasks.workunit.client.0.trial116.stdout: "snaptrimq_len": 0, 2026-02-12T18:21:22.299 INFO:tasks.workunit.client.0.trial116.stdout: "last_scrub_duration": 0, 2026-02-12T18:21:22.299 INFO:tasks.workunit.client.0.trial116.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-13T20:08:33.664778+0000", 2026-02-12T18:21:22.299 INFO:tasks.workunit.client.0.trial116.stdout: "scrub_duration": 0, 2026-02-12T18:21:22.299 INFO:tasks.workunit.client.0.trial116.stdout: "objects_trimmed": 0, 2026-02-12T18:21:22.299 INFO:tasks.workunit.client.0.trial116.stdout: "snaptrim_duration": 0, 2026-02-12T18:21:22.299 INFO:tasks.workunit.client.0.trial116.stdout: "stat_sum": { 2026-02-12T18:21:22.299 INFO:tasks.workunit.client.0.trial116.stdout: "num_bytes": 413696, 2026-02-12T18:21:22.299 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects": 101, 2026-02-12T18:21:22.299 INFO:tasks.workunit.client.0.trial116.stdout: "num_object_clones": 0, 2026-02-12T18:21:22.299 INFO:tasks.workunit.client.0.trial116.stdout: "num_object_copies": 505, 2026-02-12T18:21:22.300 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_missing_on_primary": 0, 2026-02-12T18:21:22.300 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_missing": 0, 2026-02-12T18:21:22.300 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_degraded": 101, 2026-02-12T18:21:22.300 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_misplaced": 0, 2026-02-12T18:21:22.300 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_unfound": 0, 2026-02-12T18:21:22.300 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_dirty": 101, 2026-02-12T18:21:22.300 INFO:tasks.workunit.client.0.trial116.stdout: "num_whiteouts": 0, 2026-02-12T18:21:22.300 INFO:tasks.workunit.client.0.trial116.stdout: "num_read": 0, 2026-02-12T18:21:22.300 INFO:tasks.workunit.client.0.trial116.stdout: "num_read_kb": 0, 2026-02-12T18:21:22.300 INFO:tasks.workunit.client.0.trial116.stdout: "num_write": 101, 2026-02-12T18:21:22.300 INFO:tasks.workunit.client.0.trial116.stdout: "num_write_kb": 404, 2026-02-12T18:21:22.300 INFO:tasks.workunit.client.0.trial116.stdout: "num_scrub_errors": 0, 2026-02-12T18:21:22.300 INFO:tasks.workunit.client.0.trial116.stdout: "num_shallow_scrub_errors": 0, 2026-02-12T18:21:22.300 INFO:tasks.workunit.client.0.trial116.stdout: "num_deep_scrub_errors": 0, 2026-02-12T18:21:22.300 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_recovered": 0, 2026-02-12T18:21:22.300 INFO:tasks.workunit.client.0.trial116.stdout: "num_bytes_recovered": 0, 2026-02-12T18:21:22.300 INFO:tasks.workunit.client.0.trial116.stdout: "num_keys_recovered": 0, 2026-02-12T18:21:22.300 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_omap": 0, 2026-02-12T18:21:22.300 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_hit_set_archive": 0, 2026-02-12T18:21:22.300 INFO:tasks.workunit.client.0.trial116.stdout: "num_bytes_hit_set_archive": 0, 2026-02-12T18:21:22.300 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush": 0, 2026-02-12T18:21:22.301 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush_kb": 0, 2026-02-12T18:21:22.301 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict": 0, 2026-02-12T18:21:22.301 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict_kb": 0, 2026-02-12T18:21:22.301 INFO:tasks.workunit.client.0.trial116.stdout: "num_promote": 0, 2026-02-12T18:21:22.301 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush_mode_high": 0, 2026-02-12T18:21:22.301 INFO:tasks.workunit.client.0.trial116.stdout: "num_flush_mode_low": 0, 2026-02-12T18:21:22.301 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict_mode_some": 0, 2026-02-12T18:21:22.301 INFO:tasks.workunit.client.0.trial116.stdout: "num_evict_mode_full": 0, 2026-02-12T18:21:22.301 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_pinned": 0, 2026-02-12T18:21:22.301 INFO:tasks.workunit.client.0.trial116.stdout: "num_legacy_snapsets": 0, 2026-02-12T18:21:22.301 INFO:tasks.workunit.client.0.trial116.stdout: "num_large_omap_objects": 0, 2026-02-12T18:21:22.301 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_manifest": 0, 2026-02-12T18:21:22.301 INFO:tasks.workunit.client.0.trial116.stdout: "num_omap_bytes": 0, 2026-02-12T18:21:22.301 INFO:tasks.workunit.client.0.trial116.stdout: "num_omap_keys": 0, 2026-02-12T18:21:22.301 INFO:tasks.workunit.client.0.trial116.stdout: "num_objects_repaired": 0 2026-02-12T18:21:22.301 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:21:22.301 INFO:tasks.workunit.client.0.trial116.stdout: "up": [ 2026-02-12T18:21:22.301 INFO:tasks.workunit.client.0.trial116.stdout: 3, 2026-02-12T18:21:22.301 INFO:tasks.workunit.client.0.trial116.stdout: 1, 2026-02-12T18:21:22.301 INFO:tasks.workunit.client.0.trial116.stdout: 4, 2026-02-12T18:21:22.302 INFO:tasks.workunit.client.0.trial116.stdout: 0, 2026-02-12T18:21:22.302 INFO:tasks.workunit.client.0.trial116.stdout: 2 2026-02-12T18:21:22.302 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:21:22.302 INFO:tasks.workunit.client.0.trial116.stdout: "acting": [ 2026-02-12T18:21:22.302 INFO:tasks.workunit.client.0.trial116.stdout: 3, 2026-02-12T18:21:22.302 INFO:tasks.workunit.client.0.trial116.stdout: 1, 2026-02-12T18:21:22.302 INFO:tasks.workunit.client.0.trial116.stdout: 4, 2026-02-12T18:21:22.302 INFO:tasks.workunit.client.0.trial116.stdout: 0, 2026-02-12T18:21:22.302 INFO:tasks.workunit.client.0.trial116.stdout: 2 2026-02-12T18:21:22.302 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:21:22.302 INFO:tasks.workunit.client.0.trial116.stdout: "avail_no_missing": [ 2026-02-12T18:21:22.302 INFO:tasks.workunit.client.0.trial116.stdout: "3(0)", 2026-02-12T18:21:22.302 INFO:tasks.workunit.client.0.trial116.stdout: "0(3)", 2026-02-12T18:21:22.302 INFO:tasks.workunit.client.0.trial116.stdout: "1(1)", 2026-02-12T18:21:22.302 INFO:tasks.workunit.client.0.trial116.stdout: "4(2)" 2026-02-12T18:21:22.302 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:21:22.302 INFO:tasks.workunit.client.0.trial116.stdout: "object_location_counts": [ 2026-02-12T18:21:22.302 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:21:22.302 INFO:tasks.workunit.client.0.trial116.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-12T18:21:22.302 INFO:tasks.workunit.client.0.trial116.stdout: "objects": 101 2026-02-12T18:21:22.302 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:21:22.303 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:21:22.303 INFO:tasks.workunit.client.0.trial116.stdout: "blocked_by": [], 2026-02-12T18:21:22.303 INFO:tasks.workunit.client.0.trial116.stdout: "up_primary": 3, 2026-02-12T18:21:22.303 INFO:tasks.workunit.client.0.trial116.stdout: "acting_primary": 3, 2026-02-12T18:21:22.303 INFO:tasks.workunit.client.0.trial116.stdout: "purged_snaps": [] 2026-02-12T18:21:22.303 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:21:22.303 INFO:tasks.workunit.client.0.trial116.stdout: "empty": 0, 2026-02-12T18:21:22.303 INFO:tasks.workunit.client.0.trial116.stdout: "dne": 0, 2026-02-12T18:21:22.303 INFO:tasks.workunit.client.0.trial116.stdout: "incomplete": 0, 2026-02-12T18:21:22.303 INFO:tasks.workunit.client.0.trial116.stdout: "last_epoch_started": 53, 2026-02-12T18:21:22.303 INFO:tasks.workunit.client.0.trial116.stdout: "hit_set_history": { 2026-02-12T18:21:22.303 INFO:tasks.workunit.client.0.trial116.stdout: "current_last_update": "0'0", 2026-02-12T18:21:22.303 INFO:tasks.workunit.client.0.trial116.stdout: "history": [] 2026-02-12T18:21:22.303 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:21:22.303 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:21:22.303 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:21:22.303 INFO:tasks.workunit.client.0.trial116.stdout: "recovery_state": [ 2026-02-12T18:21:22.303 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:21:22.303 INFO:tasks.workunit.client.0.trial116.stdout: "name": "Started/Primary/Active", 2026-02-12T18:21:22.303 INFO:tasks.workunit.client.0.trial116.stdout: "enter_time": "2026-02-12T18:21:06.699930+0000", 2026-02-12T18:21:22.303 INFO:tasks.workunit.client.0.trial116.stdout: "might_have_unfound": [ 2026-02-12T18:21:22.303 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:21:22.304 INFO:tasks.workunit.client.0.trial116.stdout: "osd": "0(3)", 2026-02-12T18:21:22.304 INFO:tasks.workunit.client.0.trial116.stdout: "status": "already probed" 2026-02-12T18:21:22.304 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:21:22.304 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:21:22.304 INFO:tasks.workunit.client.0.trial116.stdout: "osd": "1(1)", 2026-02-12T18:21:22.304 INFO:tasks.workunit.client.0.trial116.stdout: "status": "already probed" 2026-02-12T18:21:22.304 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:21:22.304 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:21:22.304 INFO:tasks.workunit.client.0.trial116.stdout: "osd": "2(4)", 2026-02-12T18:21:22.304 INFO:tasks.workunit.client.0.trial116.stdout: "status": "already probed" 2026-02-12T18:21:22.304 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:21:22.304 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:21:22.304 INFO:tasks.workunit.client.0.trial116.stdout: "osd": "4(2)", 2026-02-12T18:21:22.304 INFO:tasks.workunit.client.0.trial116.stdout: "status": "already probed" 2026-02-12T18:21:22.304 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:21:22.304 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:21:22.304 INFO:tasks.workunit.client.0.trial116.stdout: "recovery_progress": { 2026-02-12T18:21:22.304 INFO:tasks.workunit.client.0.trial116.stdout: "backfill_targets": [], 2026-02-12T18:21:22.304 INFO:tasks.workunit.client.0.trial116.stdout: "waiting_on_backfill": [], 2026-02-12T18:21:22.304 INFO:tasks.workunit.client.0.trial116.stdout: "last_backfill_started": "MIN", 2026-02-12T18:21:22.304 INFO:tasks.workunit.client.0.trial116.stdout: "backfill_info": { 2026-02-12T18:21:22.304 INFO:tasks.workunit.client.0.trial116.stdout: "begin": "MIN", 2026-02-12T18:21:22.304 INFO:tasks.workunit.client.0.trial116.stdout: "end": "MIN", 2026-02-12T18:21:22.305 INFO:tasks.workunit.client.0.trial116.stdout: "objects": [] 2026-02-12T18:21:22.305 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:21:22.305 INFO:tasks.workunit.client.0.trial116.stdout: "peer_backfill_info": [], 2026-02-12T18:21:22.305 INFO:tasks.workunit.client.0.trial116.stdout: "backfills_in_flight": [], 2026-02-12T18:21:22.305 INFO:tasks.workunit.client.0.trial116.stdout: "recovering": [], 2026-02-12T18:21:22.305 INFO:tasks.workunit.client.0.trial116.stdout: "pg_backend": { 2026-02-12T18:21:22.305 INFO:tasks.workunit.client.0.trial116.stdout: "recovery_ops": [], 2026-02-12T18:21:22.305 INFO:tasks.workunit.client.0.trial116.stdout: "read_ops": [] 2026-02-12T18:21:22.305 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:21:22.305 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:21:22.305 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:21:22.305 INFO:tasks.workunit.client.0.trial116.stdout: { 2026-02-12T18:21:22.305 INFO:tasks.workunit.client.0.trial116.stdout: "name": "Started", 2026-02-12T18:21:22.305 INFO:tasks.workunit.client.0.trial116.stdout: "enter_time": "2026-02-12T18:21:06.598054+0000" 2026-02-12T18:21:22.305 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:21:22.305 INFO:tasks.workunit.client.0.trial116.stdout: ], 2026-02-12T18:21:22.305 INFO:tasks.workunit.client.0.trial116.stdout: "scrubber": { 2026-02-12T18:21:22.305 INFO:tasks.workunit.client.0.trial116.stdout: "active": false, 2026-02-12T18:21:22.305 INFO:tasks.workunit.client.0.trial116.stdout: "must_scrub": false, 2026-02-12T18:21:22.305 INFO:tasks.workunit.client.0.trial116.stdout: "must_deep_scrub": false, 2026-02-12T18:21:22.305 INFO:tasks.workunit.client.0.trial116.stdout: "must_repair": false, 2026-02-12T18:21:22.305 INFO:tasks.workunit.client.0.trial116.stdout: "sched_time": "2026-02-13T20:08:33.664778+0000", 2026-02-12T18:21:22.305 INFO:tasks.workunit.client.0.trial116.stdout: "schedule": "not registered for scrubbing", 2026-02-12T18:21:22.306 INFO:tasks.workunit.client.0.trial116.stdout: "urgency": "periodic-regular", 2026-02-12T18:21:22.306 INFO:tasks.workunit.client.0.trial116.stdout: "shallow-target": { 2026-02-12T18:21:22.306 INFO:tasks.workunit.client.0.trial116.stdout: "pgid": "2.0s0", 2026-02-12T18:21:22.306 INFO:tasks.workunit.client.0.trial116.stdout: "level": "shallow", 2026-02-12T18:21:22.306 INFO:tasks.workunit.client.0.trial116.stdout: "urgency": "periodic-regular", 2026-02-12T18:21:22.306 INFO:tasks.workunit.client.0.trial116.stdout: "sched_time": "2026-02-13T20:08:33.664778+0000", 2026-02-12T18:21:22.306 INFO:tasks.workunit.client.0.trial116.stdout: "orig_sched_time": "2026-02-13T20:08:33.664778+0000", 2026-02-12T18:21:22.306 INFO:tasks.workunit.client.0.trial116.stdout: "last_issue": "ok", 2026-02-12T18:21:22.306 INFO:tasks.workunit.client.0.trial116.stdout: "forced": false, 2026-02-12T18:21:22.306 INFO:tasks.workunit.client.0.trial116.stdout: "eligible": false, 2026-02-12T18:21:22.306 INFO:tasks.workunit.client.0.trial116.stdout: "queued": false, 2026-02-12T18:21:22.306 INFO:tasks.workunit.client.0.trial116.stdout: "active": false 2026-02-12T18:21:22.306 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:21:22.306 INFO:tasks.workunit.client.0.trial116.stdout: "deep-target": { 2026-02-12T18:21:22.306 INFO:tasks.workunit.client.0.trial116.stdout: "pgid": "2.0s0", 2026-02-12T18:21:22.306 INFO:tasks.workunit.client.0.trial116.stdout: "level": "deep", 2026-02-12T18:21:22.306 INFO:tasks.workunit.client.0.trial116.stdout: "urgency": "periodic-regular", 2026-02-12T18:21:22.306 INFO:tasks.workunit.client.0.trial116.stdout: "sched_time": "2026-02-20T04:23:13.425840+0000", 2026-02-12T18:21:22.306 INFO:tasks.workunit.client.0.trial116.stdout: "orig_sched_time": "2026-02-20T04:23:13.425840+0000", 2026-02-12T18:21:22.306 INFO:tasks.workunit.client.0.trial116.stdout: "last_issue": "ok", 2026-02-12T18:21:22.306 INFO:tasks.workunit.client.0.trial116.stdout: "forced": false, 2026-02-12T18:21:22.306 INFO:tasks.workunit.client.0.trial116.stdout: "eligible": false, 2026-02-12T18:21:22.306 INFO:tasks.workunit.client.0.trial116.stdout: "queued": false, 2026-02-12T18:21:22.307 INFO:tasks.workunit.client.0.trial116.stdout: "active": false 2026-02-12T18:21:22.307 INFO:tasks.workunit.client.0.trial116.stdout: } 2026-02-12T18:21:22.307 INFO:tasks.workunit.client.0.trial116.stdout: }, 2026-02-12T18:21:22.307 INFO:tasks.workunit.client.0.trial116.stdout: "agent_state": {} 2026-02-12T18:21:22.307 INFO:tasks.workunit.client.0.trial116.stdout:} 2026-02-12T18:21:22.307 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:684: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-12T18:21:22.307 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:684: TEST_ec_recovery_unfound: grep -q obj75 2026-02-12T18:21:22.318 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-12T18:21:22.318 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: jq .available_might_have_unfound 2026-02-12T18:21:22.382 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: check=true 2026-02-12T18:21:22.382 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: test true == true 2026-02-12T18:21:22.383 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:689: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-12T18:21:22.383 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:689: TEST_ec_recovery_unfound: jq '.might_have_unfound | length' 2026-02-12T18:21:22.448 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:689: TEST_ec_recovery_unfound: check=0 2026-02-12T18:21:22.448 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:690: TEST_ec_recovery_unfound: test 0 == 0 2026-02-12T18:21:22.448 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:693: TEST_ec_recovery_unfound: timeout 5 rados -p pool-jerasure get obj75 td/test-erasure-eio/CHECK 2026-02-12T18:21:27.452 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:694: TEST_ec_recovery_unfound: test 124 = 124 2026-02-12T18:21:27.452 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:696: TEST_ec_recovery_unfound: ceph pg 2.0 mark_unfound_lost delete 2026-02-12T18:21:27.515 INFO:tasks.workunit.client.0.trial116.stderr:pg has no unfound objects 2026-02-12T18:21:27.520 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:698: TEST_ec_recovery_unfound: wait_for_clean 2026-02-12T18:21:27.520 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:21:27.520 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:21:27.520 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:21:27.521 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:21:27.521 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:21:27.521 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:21:27.522 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:21:27.522 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:21:27.522 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:21:27.606 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:21:27.606 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:21:27.606 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:21:27.606 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:21:27.606 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:21:27.607 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:21:27.836 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:21:27.837 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:21:27.837 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:21:27.837 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:21:27.837 INFO:tasks.workunit.client.0.trial116.stderr:4' 2026-02-12T18:21:27.837 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:21:27.837 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:21:27.837 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:21:27.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803789 2026-02-12T18:21:27.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803789 2026-02-12T18:21:27.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803789' 2026-02-12T18:21:27.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:21:27.902 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:21:27.966 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574862 2026-02-12T18:21:27.966 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574862 2026-02-12T18:21:27.966 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803789 1-55834574862' 2026-02-12T18:21:27.966 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:21:27.966 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:21:28.032 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=206158430215 2026-02-12T18:21:28.032 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 206158430215 2026-02-12T18:21:28.032 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803789 1-55834574862 2-206158430215' 2026-02-12T18:21:28.032 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:21:28.032 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:21:28.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149708 2026-02-12T18:21:28.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149708 2026-02-12T18:21:28.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803789 1-55834574862 2-206158430215 3-111669149708' 2026-02-12T18:21:28.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:21:28.098 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-12T18:21:28.162 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855371 2026-02-12T18:21:28.162 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855371 2026-02-12T18:21:28.162 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803789 1-55834574862 2-206158430215 3-111669149708 4-150323855371' 2026-02-12T18:21:28.163 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:21:28.163 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803789 2026-02-12T18:21:28.163 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:21:28.164 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:21:28.164 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803789 2026-02-12T18:21:28.164 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:21:28.165 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803789 2026-02-12T18:21:28.165 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803789' 2026-02-12T18:21:28.165 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.0 seq 25769803789 2026-02-12T18:21:28.165 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:21:28.393 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803787 -lt 25769803789 2026-02-12T18:21:28.393 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:21:29.394 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:21:29.395 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:21:29.626 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803789 -lt 25769803789 2026-02-12T18:21:29.626 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:21:29.627 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574862 2026-02-12T18:21:29.627 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:21:29.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:21:29.628 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574862 2026-02-12T18:21:29.628 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:21:29.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574862 2026-02-12T18:21:29.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574862' 2026-02-12T18:21:29.630 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.1 seq 55834574862 2026-02-12T18:21:29.630 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:21:29.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574862 -lt 55834574862 2026-02-12T18:21:29.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:21:29.860 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-206158430215 2026-02-12T18:21:29.860 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:21:29.860 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:21:29.861 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-206158430215 2026-02-12T18:21:29.861 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:21:29.862 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=206158430215 2026-02-12T18:21:29.862 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 206158430215' 2026-02-12T18:21:29.862 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.2 seq 206158430215 2026-02-12T18:21:29.863 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:21:30.101 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 206158430215 -lt 206158430215 2026-02-12T18:21:30.101 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:21:30.102 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149708 2026-02-12T18:21:30.102 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:21:30.103 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:21:30.103 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149708 2026-02-12T18:21:30.104 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:21:30.105 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149708 2026-02-12T18:21:30.105 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149708' 2026-02-12T18:21:30.105 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.3 seq 111669149708 2026-02-12T18:21:30.105 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:21:30.330 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149708 -lt 111669149708 2026-02-12T18:21:30.330 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:21:30.330 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-150323855371 2026-02-12T18:21:30.330 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:21:30.331 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-12T18:21:30.332 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-150323855371 2026-02-12T18:21:30.332 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:21:30.333 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=150323855371 2026-02-12T18:21:30.333 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 150323855371' 2026-02-12T18:21:30.333 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.4 seq 150323855371 2026-02-12T18:21:30.333 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-12T18:21:30.565 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 150323855371 -lt 150323855371 2026-02-12T18:21:30.566 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:21:30.566 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:21:30.566 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:21:30.878 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:21:30.878 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:21:30.878 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:21:30.879 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:21:30.879 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:21:30.879 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:21:30.879 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:21:30.879 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:21:31.110 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:21:31.111 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:21:31.111 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:21:31.111 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:21:31.424 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:21:31.424 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:21:31.424 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:21:31.425 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: seq 1 100 2026-02-12T18:21:31.425 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.425 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj1 = obj75 ']' 2026-02-12T18:21:31.425 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj1 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj2 = obj75 ']' 2026-02-12T18:21:31.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj2 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.460 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.461 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.461 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj3 = obj75 ']' 2026-02-12T18:21:31.461 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj3 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.477 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.478 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.478 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj4 = obj75 ']' 2026-02-12T18:21:31.479 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj4 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.493 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.494 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.495 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj5 = obj75 ']' 2026-02-12T18:21:31.495 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj5 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.510 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj6 = obj75 ']' 2026-02-12T18:21:31.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj6 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.527 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.528 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.528 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj7 = obj75 ']' 2026-02-12T18:21:31.528 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj7 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.545 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.545 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj8 = obj75 ']' 2026-02-12T18:21:31.545 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj8 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.560 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.561 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.561 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj9 = obj75 ']' 2026-02-12T18:21:31.561 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj9 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.577 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.578 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.578 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj10 = obj75 ']' 2026-02-12T18:21:31.578 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj10 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.594 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.595 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.595 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj11 = obj75 ']' 2026-02-12T18:21:31.595 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj11 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.611 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.612 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.612 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj12 = obj75 ']' 2026-02-12T18:21:31.612 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj12 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj13 = obj75 ']' 2026-02-12T18:21:31.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj13 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.645 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.646 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.646 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj14 = obj75 ']' 2026-02-12T18:21:31.647 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj14 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.662 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.663 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj15 = obj75 ']' 2026-02-12T18:21:31.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj15 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.679 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.680 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.680 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj16 = obj75 ']' 2026-02-12T18:21:31.680 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj16 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.695 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.696 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.696 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj17 = obj75 ']' 2026-02-12T18:21:31.696 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj17 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.713 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.714 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.714 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj18 = obj75 ']' 2026-02-12T18:21:31.714 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj18 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.730 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj19 = obj75 ']' 2026-02-12T18:21:31.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj19 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.746 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.747 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.747 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj20 = obj75 ']' 2026-02-12T18:21:31.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj20 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.764 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.764 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj21 = obj75 ']' 2026-02-12T18:21:31.764 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj21 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.779 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.780 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.781 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj22 = obj75 ']' 2026-02-12T18:21:31.781 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj22 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.796 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj23 = obj75 ']' 2026-02-12T18:21:31.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj23 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.813 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.814 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.815 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj24 = obj75 ']' 2026-02-12T18:21:31.815 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj24 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.830 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.831 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.831 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj25 = obj75 ']' 2026-02-12T18:21:31.831 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj25 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.846 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.847 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.847 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj26 = obj75 ']' 2026-02-12T18:21:31.848 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj26 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.863 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.864 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.864 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj27 = obj75 ']' 2026-02-12T18:21:31.864 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj27 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.880 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj28 = obj75 ']' 2026-02-12T18:21:31.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj28 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.898 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.899 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.899 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj29 = obj75 ']' 2026-02-12T18:21:31.899 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj29 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.915 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.915 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.916 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj30 = obj75 ']' 2026-02-12T18:21:31.916 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj30 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.931 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.932 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.932 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj31 = obj75 ']' 2026-02-12T18:21:31.932 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj31 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.948 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.949 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.949 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj32 = obj75 ']' 2026-02-12T18:21:31.949 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj32 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.966 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.967 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.967 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj33 = obj75 ']' 2026-02-12T18:21:31.967 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj33 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.982 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:31.983 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:31.983 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj34 = obj75 ']' 2026-02-12T18:21:31.984 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj34 td/test-erasure-eio/CHECK 2026-02-12T18:21:31.999 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.000 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.000 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj35 = obj75 ']' 2026-02-12T18:21:32.000 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj35 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.016 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.017 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.017 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj36 = obj75 ']' 2026-02-12T18:21:32.017 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj36 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.033 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.034 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.034 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj37 = obj75 ']' 2026-02-12T18:21:32.034 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj37 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.050 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.050 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.051 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj38 = obj75 ']' 2026-02-12T18:21:32.051 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj38 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.066 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.067 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.067 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj39 = obj75 ']' 2026-02-12T18:21:32.067 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj39 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.083 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.084 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.084 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj40 = obj75 ']' 2026-02-12T18:21:32.084 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj40 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.099 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.100 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.100 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj41 = obj75 ']' 2026-02-12T18:21:32.100 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj41 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.116 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.117 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.117 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj42 = obj75 ']' 2026-02-12T18:21:32.117 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj42 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.134 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj43 = obj75 ']' 2026-02-12T18:21:32.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj43 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.151 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.152 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.152 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj44 = obj75 ']' 2026-02-12T18:21:32.152 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj44 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.169 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.169 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.170 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj45 = obj75 ']' 2026-02-12T18:21:32.170 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj45 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.185 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.186 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.186 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj46 = obj75 ']' 2026-02-12T18:21:32.186 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj46 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.202 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.327 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.327 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj47 = obj75 ']' 2026-02-12T18:21:32.327 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj47 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.327 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.327 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.327 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj48 = obj75 ']' 2026-02-12T18:21:32.327 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj48 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.327 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.327 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.327 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj49 = obj75 ']' 2026-02-12T18:21:32.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj49 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj50 = obj75 ']' 2026-02-12T18:21:32.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj50 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj51 = obj75 ']' 2026-02-12T18:21:32.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj51 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj52 = obj75 ']' 2026-02-12T18:21:32.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj52 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj53 = obj75 ']' 2026-02-12T18:21:32.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj53 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj54 = obj75 ']' 2026-02-12T18:21:32.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj54 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.334 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.335 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.335 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj55 = obj75 ']' 2026-02-12T18:21:32.335 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj55 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.351 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.352 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.352 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj56 = obj75 ']' 2026-02-12T18:21:32.352 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj56 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.368 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.368 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.369 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj57 = obj75 ']' 2026-02-12T18:21:32.369 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj57 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.384 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj58 = obj75 ']' 2026-02-12T18:21:32.386 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj58 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.401 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.402 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.402 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj59 = obj75 ']' 2026-02-12T18:21:32.402 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj59 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.420 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj60 = obj75 ']' 2026-02-12T18:21:32.420 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj60 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.435 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.436 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.436 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj61 = obj75 ']' 2026-02-12T18:21:32.436 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj61 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.452 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.453 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.453 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj62 = obj75 ']' 2026-02-12T18:21:32.453 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj62 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.469 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.469 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.469 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj63 = obj75 ']' 2026-02-12T18:21:32.470 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj63 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.485 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.486 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.486 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj64 = obj75 ']' 2026-02-12T18:21:32.486 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj64 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.503 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.504 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.504 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj65 = obj75 ']' 2026-02-12T18:21:32.504 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj65 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.520 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.521 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.521 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj66 = obj75 ']' 2026-02-12T18:21:32.522 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj66 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.537 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.538 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.539 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj67 = obj75 ']' 2026-02-12T18:21:32.539 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj67 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.555 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.556 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.556 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj68 = obj75 ']' 2026-02-12T18:21:32.556 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj68 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.571 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.572 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.572 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj69 = obj75 ']' 2026-02-12T18:21:32.572 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj69 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.588 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj70 = obj75 ']' 2026-02-12T18:21:32.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj70 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.606 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.607 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.607 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj71 = obj75 ']' 2026-02-12T18:21:32.607 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj71 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.623 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.624 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.624 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj72 = obj75 ']' 2026-02-12T18:21:32.624 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj72 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.640 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.641 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.641 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj73 = obj75 ']' 2026-02-12T18:21:32.641 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj73 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.657 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.658 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.658 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj74 = obj75 ']' 2026-02-12T18:21:32.658 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj74 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.675 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.676 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj75 = obj75 ']' 2026-02-12T18:21:32.676 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:704: TEST_ec_recovery_unfound: rados -p pool-jerasure get obj75 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.691 INFO:tasks.workunit.client.0.trial116.stderr:error getting pool-jerasure/obj75: (2) No such file or directory 2026-02-12T18:21:32.692 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.692 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj76 = obj75 ']' 2026-02-12T18:21:32.692 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj76 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.707 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj77 = obj75 ']' 2026-02-12T18:21:32.709 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj77 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.725 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.725 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.726 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj78 = obj75 ']' 2026-02-12T18:21:32.726 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj78 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.742 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.743 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.743 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj79 = obj75 ']' 2026-02-12T18:21:32.743 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj79 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.760 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.761 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.761 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj80 = obj75 ']' 2026-02-12T18:21:32.761 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj80 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.777 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.778 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.778 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj81 = obj75 ']' 2026-02-12T18:21:32.778 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj81 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.794 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.795 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.795 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj82 = obj75 ']' 2026-02-12T18:21:32.795 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj82 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.810 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.811 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.812 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj83 = obj75 ']' 2026-02-12T18:21:32.812 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj83 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj84 = obj75 ']' 2026-02-12T18:21:32.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj84 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.844 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.845 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.845 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj85 = obj75 ']' 2026-02-12T18:21:32.845 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj85 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.860 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.861 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.862 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj86 = obj75 ']' 2026-02-12T18:21:32.862 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj86 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.877 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.878 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.878 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj87 = obj75 ']' 2026-02-12T18:21:32.878 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj87 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.894 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.895 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.895 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj88 = obj75 ']' 2026-02-12T18:21:32.895 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj88 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.911 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.912 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.912 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj89 = obj75 ']' 2026-02-12T18:21:32.912 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj89 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.927 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj90 = obj75 ']' 2026-02-12T18:21:32.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj90 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.944 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.944 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.944 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj91 = obj75 ']' 2026-02-12T18:21:32.944 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj91 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.960 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.961 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.962 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj92 = obj75 ']' 2026-02-12T18:21:32.962 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj92 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.977 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.978 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.978 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj93 = obj75 ']' 2026-02-12T18:21:32.979 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj93 td/test-erasure-eio/CHECK 2026-02-12T18:21:32.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:32.996 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:32.997 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj94 = obj75 ']' 2026-02-12T18:21:32.997 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj94 td/test-erasure-eio/CHECK 2026-02-12T18:21:33.013 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:33.014 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:33.014 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj95 = obj75 ']' 2026-02-12T18:21:33.014 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj95 td/test-erasure-eio/CHECK 2026-02-12T18:21:33.030 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:33.031 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:33.031 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj96 = obj75 ']' 2026-02-12T18:21:33.031 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj96 td/test-erasure-eio/CHECK 2026-02-12T18:21:33.047 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:33.048 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:33.048 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj97 = obj75 ']' 2026-02-12T18:21:33.049 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj97 td/test-erasure-eio/CHECK 2026-02-12T18:21:33.064 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:33.065 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:33.065 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj98 = obj75 ']' 2026-02-12T18:21:33.065 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj98 td/test-erasure-eio/CHECK 2026-02-12T18:21:33.081 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:33.082 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:33.082 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj99 = obj75 ']' 2026-02-12T18:21:33.082 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj99 td/test-erasure-eio/CHECK 2026-02-12T18:21:33.098 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:33.099 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-12T18:21:33.099 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj100 = obj75 ']' 2026-02-12T18:21:33.099 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj100 td/test-erasure-eio/CHECK 2026-02-12T18:21:33.115 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:33.115 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:711: TEST_ec_recovery_unfound: rm -f td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-12T18:21:33.116 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:713: TEST_ec_recovery_unfound: delete_erasure_coded_pool pool-jerasure 2026-02-12T18:21:33.116 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-12T18:21:33.116 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-12T18:21:33.395 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-jerasure' does not exist 2026-02-12T18:21:33.403 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-12T18:21:33.704 INFO:tasks.workunit.client.0.trial116.stderr:erasure-code-profile myprofile does not exist 2026-02-12T18:21:33.713 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-12T18:21:33.713 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-12T18:21:33.713 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:21:33.713 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-12T18:21:33.714 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:21:33.714 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:21:33.714 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:21:33.714 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:21:33.714 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:21:33.822 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:21:33.823 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:21:33.823 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:21:33.824 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:21:33.825 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:21:33.825 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:21:33.826 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:21:33.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:21:33.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:21:33.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:21:33.827 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:21:33.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:21:33.829 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:21:33.829 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-12T18:21:33.850 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:21:33.850 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:21:33.850 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:21:33.850 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76821 2026-02-12T18:21:33.853 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:21:33.853 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:21:33.853 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-12T18:21:33.854 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-12T18:21:33.854 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-12T18:21:33.854 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-12T18:21:33.854 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-12T18:21:33.854 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:21:33.854 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-12T18:21:33.854 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:21:33.854 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:21:33.855 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:21:33.855 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:21:33.855 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:21:33.855 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:21:33.855 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:21:33.856 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:21:33.856 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:21:33.856 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:21:33.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:21:33.857 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:21:33.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:21:33.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:21:33.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:21:33.857 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:21:33.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:21:33.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:21:33.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-12T18:21:33.860 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:21:33.860 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:21:33.860 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:21:33.860 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76821 2026-02-12T18:21:33.861 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:21:33.861 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:21:33.861 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-12T18:21:33.862 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T18:21:33.862 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:21:33.863 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:21:33.863 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76821 2026-02-12T18:21:33.864 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T18:21:33.865 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T18:21:33.865 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T18:21:33.865 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-12T18:21:33.865 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-12T18:21:33.865 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-12T18:21:33.865 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T18:21:33.865 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T18:21:33.865 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T18:21:33.866 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-12T18:21:33.866 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-12T18:21:33.882 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T18:21:33.883 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:21:33.883 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:21:33.883 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:21:33.883 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:21:33.883 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:21:33.883 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:21:33.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T18:21:33.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T18:21:33.907 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T18:21:33.907 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:21:33.908 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:21:33.908 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T18:21:33.908 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T18:21:33.909 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:21:33.909 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:21:33.909 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:21:33.909 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:21:33.909 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:21:33.909 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:21:33.909 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:21:33.910 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:21:33.910 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76821/ceph-mon.a.asok config get fsid 2026-02-12T18:21:33.954 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T18:21:33.955 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:21:33.955 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:21:33.955 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T18:21:33.955 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T18:21:33.955 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:21:33.955 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:21:33.955 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:21:33.955 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:21:33.956 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:21:33.956 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:21:33.956 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:21:33.956 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:21:33.956 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76821/ceph-mon.a.asok config get mon_host 2026-02-12T18:21:33.999 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-12T18:21:33.999 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-12T18:21:33.999 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T18:21:33.999 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T18:21:33.999 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T18:21:33.999 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-12T18:21:33.999 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T18:21:34.099 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T18:21:34.099 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:21:34.100 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:21:34.100 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:21:34.100 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:21:34.100 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:21:34.100 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:21:34.101 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:21:34.101 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:21:34.116 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-12T18:21:34.116 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T18:21:34.212 INFO:tasks.workunit.client.0.trial116.stderr:pool 'rbd' already exists 2026-02-12T18:21:34.219 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:21:35.221 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-12T18:21:35.222 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:21:35.222 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:21:35.222 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:21:35.222 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:21:35.222 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:21:35.222 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:21:35.222 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-12T18:21:35.222 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-mon.a.asok log flush 2026-02-12T18:21:35.261 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:21:35.266 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-12T18:21:35.267 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:21:33.897+0000 7fe9dbc0dc00 0 load: jerasure load: lrc 2026-02-12T18:21:35.268 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_single_recovery_error td/test-erasure-eio 2026-02-12T18:21:35.269 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:410: TEST_ec_single_recovery_error: local dir=td/test-erasure-eio 2026-02-12T18:21:35.269 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:411: TEST_ec_single_recovery_error: local objname=myobject 2026-02-12T18:21:35.269 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:413: TEST_ec_single_recovery_error: setup_osds 7 2026-02-12T18:21:35.269 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-12T18:21:35.269 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-12T18:21:35.269 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-12T18:21:35.269 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-12T18:21:35.273 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:21:35.273 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-12T18:21:35.273 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:21:35.273 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:21:35.273 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T18:21:35.273 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:21:35.274 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-12T18:21:35.274 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:21:35.274 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:21:35.274 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:21:35.274 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:21:35.274 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-12T18:21:35.274 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-12T18:21:35.274 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:21:35.274 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:21:35.274 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:21:35.274 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:21:35.274 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:21:35.275 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:21:35.275 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:21:35.275 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:21:35.275 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:21:35.275 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:21:35.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:21:35.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:21:35.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:21:35.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:21:35.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:21:35.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:21:35.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:21:35.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:21:35.276 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:21:35.276 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:21:35.276 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:21:35.276 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:21:35.276 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-12T18:21:35.276 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:21:35.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d5caadd4-ed9f-4ef9-8dea-aeab37a9c68d 2026-02-12T18:21:35.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 d5caadd4-ed9f-4ef9-8dea-aeab37a9c68d' 2026-02-12T18:21:35.277 INFO:tasks.workunit.client.0.trial116.stdout:add osd0 d5caadd4-ed9f-4ef9-8dea-aeab37a9c68d 2026-02-12T18:21:35.277 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:21:35.283 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAvGo5pksq2EBAAc+m2lt+jgcz7asH2acMXAg== 2026-02-12T18:21:35.283 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAvGo5pksq2EBAAc+m2lt+jgcz7asH2acMXAg=="}' 2026-02-12T18:21:35.283 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d5caadd4-ed9f-4ef9-8dea-aeab37a9c68d -i td/test-erasure-eio/0/new.json 2026-02-12T18:21:35.388 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:21:35.394 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-12T18:21:35.395 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAvGo5pksq2EBAAc+m2lt+jgcz7asH2acMXAg== --osd-uuid d5caadd4-ed9f-4ef9-8dea-aeab37a9c68d 2026-02-12T18:21:35.411 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:35.409+0000 7f2805e0c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:35.413 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:35.410+0000 7f2805e0c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:35.413 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:35.411+0000 7f2805e0c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:35.414 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:35.411+0000 7f2805e0c780 -1 bdev(0x3ac8c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-12T18:21:35.414 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:35.411+0000 7f2805e0c780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-12T18:21:35.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-12T18:21:35.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:21:35.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T18:21:35.859 INFO:tasks.workunit.client.0.trial116.stdout:adding osd0 key to auth repository 2026-02-12T18:21:35.860 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:21:35.962 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T18:21:35.962 INFO:tasks.workunit.client.0.trial116.stdout:start osd.0 2026-02-12T18:21:35.962 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:21:35.962 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:21:35.962 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:21:35.962 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:21:35.976 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:35.974+0000 7fa8aa043780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:35.978 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:35.975+0000 7fa8aa043780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:35.979 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:35.976+0000 7fa8aa043780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:36.058 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T18:21:36.058 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:21:36.058 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T18:21:36.058 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:21:36.058 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:21:36.058 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:21:36.058 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:21:36.058 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:21:36.058 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:21:36.058 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:21:36.155 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:21:36.188 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:36.186+0000 7fa8aa043780 -1 Falling back to public interface 2026-02-12T18:21:36.434 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:36.431+0000 7fa8aa043780 -1 osd.0 0 log_to_monitors true 2026-02-12T18:21:37.157 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:21:37.157 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:21:37.158 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:21:37.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:21:37.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:21:37.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:21:37.395 INFO:tasks.workunit.client.0.trial116.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3152419873,v1:127.0.0.1:6803/3152419873] [v2:127.0.0.1:6804/3152419873,v1:127.0.0.1:6805/3152419873] exists,up d5caadd4-ed9f-4ef9-8dea-aeab37a9c68d 2026-02-12T18:21:37.395 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:21:37.395 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:21:37.396 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:21:37.396 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:21:37.396 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-12T18:21:37.396 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:21:37.396 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:21:37.396 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-12T18:21:37.396 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:21:37.396 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-12T18:21:37.396 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:21:37.397 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:21:37.397 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:21:37.397 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:21:37.397 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-12T18:21:37.397 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-12T18:21:37.397 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:21:37.397 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:21:37.398 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:21:37.398 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:21:37.398 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:21:37.398 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:21:37.398 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:21:37.398 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:21:37.398 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:21:37.398 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:21:37.398 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:21:37.398 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:21:37.398 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:21:37.398 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:21:37.398 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:21:37.398 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:21:37.398 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:21:37.398 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:21:37.398 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:21:37.398 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:21:37.398 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:21:37.398 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:21:37.398 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-12T18:21:37.398 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:21:37.399 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=32e4effd-3492-4a08-908a-e3d022c6f1d8 2026-02-12T18:21:37.399 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 32e4effd-3492-4a08-908a-e3d022c6f1d8' 2026-02-12T18:21:37.399 INFO:tasks.workunit.client.0.trial116.stdout:add osd1 32e4effd-3492-4a08-908a-e3d022c6f1d8 2026-02-12T18:21:37.399 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:21:37.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAxGo5p5nk1GBAAva4K8T04wfUL7XRFdbXA/w== 2026-02-12T18:21:37.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAxGo5p5nk1GBAAva4K8T04wfUL7XRFdbXA/w=="}' 2026-02-12T18:21:37.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 32e4effd-3492-4a08-908a-e3d022c6f1d8 -i td/test-erasure-eio/1/new.json 2026-02-12T18:21:37.641 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:21:37.649 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-12T18:21:37.650 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAxGo5p5nk1GBAAva4K8T04wfUL7XRFdbXA/w== --osd-uuid 32e4effd-3492-4a08-908a-e3d022c6f1d8 2026-02-12T18:21:37.666 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:37.663+0000 7ff52202c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:37.667 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:37.665+0000 7ff52202c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:37.668 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:37.665+0000 7ff52202c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:37.669 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:37.666+0000 7ff52202c780 -1 bdev(0x30b0c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-12T18:21:37.669 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:37.666+0000 7ff52202c780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-12T18:21:37.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-12T18:21:37.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:21:37.996 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-12T18:21:37.996 INFO:tasks.workunit.client.0.trial116.stdout:adding osd1 key to auth repository 2026-02-12T18:21:37.997 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:21:38.322 INFO:tasks.workunit.client.0.trial116.stdout:start osd.1 2026-02-12T18:21:38.323 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-12T18:21:38.323 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:21:38.324 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:21:38.324 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:21:38.324 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:21:38.337 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:38.334+0000 7f87f2be3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:38.338 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:38.335+0000 7f87f2be3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:38.339 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:38.336+0000 7f87f2be3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:38.503 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:38.501+0000 7f87f2be3780 -1 Falling back to public interface 2026-02-12T18:21:38.550 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-12T18:21:38.550 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:21:38.550 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T18:21:38.550 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:21:38.551 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:21:38.551 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:21:38.551 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:21:38.552 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:21:38.552 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:21:38.552 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:21:38.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:21:38.801 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:38.799+0000 7f87f2be3780 -1 osd.1 0 log_to_monitors true 2026-02-12T18:21:39.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:21:39.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:21:39.783 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:21:39.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:21:39.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:21:39.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:21:40.009 INFO:tasks.workunit.client.0.trial116.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/872078660,v1:127.0.0.1:6811/872078660] [v2:127.0.0.1:6812/872078660,v1:127.0.0.1:6813/872078660] exists,up 32e4effd-3492-4a08-908a-e3d022c6f1d8 2026-02-12T18:21:40.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:21:40.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:21:40.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:21:40.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:21:40.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-12T18:21:40.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:21:40.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:21:40.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-12T18:21:40.011 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:21:40.011 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-12T18:21:40.011 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:21:40.011 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:21:40.011 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:21:40.011 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:21:40.011 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-12T18:21:40.011 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-12T18:21:40.011 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:21:40.011 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:21:40.011 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:21:40.011 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:21:40.011 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:21:40.011 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:21:40.011 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:21:40.011 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:21:40.011 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:21:40.012 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:21:40.012 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:21:40.012 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:21:40.012 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:21:40.012 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:21:40.012 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:21:40.012 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:21:40.012 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:21:40.012 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:21:40.012 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:21:40.012 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:21:40.012 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:21:40.012 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:21:40.012 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-12T18:21:40.013 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:21:40.013 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=415468f8-31b7-4b39-856f-9eb11c86c8fc 2026-02-12T18:21:40.013 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 415468f8-31b7-4b39-856f-9eb11c86c8fc' 2026-02-12T18:21:40.013 INFO:tasks.workunit.client.0.trial116.stdout:add osd2 415468f8-31b7-4b39-856f-9eb11c86c8fc 2026-02-12T18:21:40.013 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:21:40.022 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA0Go5p4bsuARAARLKIrBMEtf/kiVokHgonLA== 2026-02-12T18:21:40.022 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA0Go5p4bsuARAARLKIrBMEtf/kiVokHgonLA=="}' 2026-02-12T18:21:40.022 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 415468f8-31b7-4b39-856f-9eb11c86c8fc -i td/test-erasure-eio/2/new.json 2026-02-12T18:21:40.253 INFO:tasks.workunit.client.0.trial116.stdout:2 2026-02-12T18:21:40.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-12T18:21:40.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA0Go5p4bsuARAARLKIrBMEtf/kiVokHgonLA== --osd-uuid 415468f8-31b7-4b39-856f-9eb11c86c8fc 2026-02-12T18:21:40.279 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:40.276+0000 7fbd98de1780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:40.281 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:40.278+0000 7fbd98de1780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:40.281 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:40.279+0000 7fbd98de1780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:40.282 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:40.279+0000 7fbd98de1780 -1 bdev(0x322ec00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-12T18:21:40.282 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:40.279+0000 7fbd98de1780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-12T18:21:40.602 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-12T18:21:40.602 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:21:40.603 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-12T18:21:40.603 INFO:tasks.workunit.client.0.trial116.stdout:adding osd2 key to auth repository 2026-02-12T18:21:40.603 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:21:40.936 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-12T18:21:40.937 INFO:tasks.workunit.client.0.trial116.stdout:start osd.2 2026-02-12T18:21:40.937 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:21:40.937 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:21:40.937 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:21:40.937 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:21:40.951 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:40.949+0000 7f66b0c60780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:40.953 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:40.950+0000 7f66b0c60780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:40.954 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:40.951+0000 7f66b0c60780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:41.082 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:41.080+0000 7f66b0c60780 -1 Falling back to public interface 2026-02-12T18:21:41.163 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-12T18:21:41.163 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:21:41.163 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T18:21:41.164 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:21:41.164 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:21:41.164 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:21:41.164 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:21:41.164 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:21:41.164 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:21:41.164 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:21:41.391 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:21:41.419 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:41.416+0000 7f66b0c60780 -1 osd.2 0 log_to_monitors true 2026-02-12T18:21:42.392 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:21:42.392 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:21:42.393 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:21:42.394 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:21:42.394 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:21:42.394 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:21:42.623 INFO:tasks.workunit.client.0.trial116.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1338884662,v1:127.0.0.1:6819/1338884662] [v2:127.0.0.1:6820/1338884662,v1:127.0.0.1:6821/1338884662] exists,up 415468f8-31b7-4b39-856f-9eb11c86c8fc 2026-02-12T18:21:42.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:21:42.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:21:42.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:21:42.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:21:42.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-12T18:21:42.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:21:42.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:21:42.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-12T18:21:42.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:21:42.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-12T18:21:42.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:21:42.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:21:42.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:21:42.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:21:42.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-12T18:21:42.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-12T18:21:42.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:21:42.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:21:42.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:21:42.628 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:21:42.629 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:21:42.629 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:21:42.629 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:21:42.629 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:21:42.629 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:21:42.629 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:21:42.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:21:42.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:21:42.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:21:42.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:21:42.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:21:42.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:21:42.630 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:21:42.630 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:21:42.630 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:21:42.630 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:21:42.630 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:21:42.630 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:21:42.630 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-12T18:21:42.630 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:21:42.630 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=956db082-779b-4d45-89ba-86e1eea4808c 2026-02-12T18:21:42.630 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 956db082-779b-4d45-89ba-86e1eea4808c' 2026-02-12T18:21:42.631 INFO:tasks.workunit.client.0.trial116.stdout:add osd3 956db082-779b-4d45-89ba-86e1eea4808c 2026-02-12T18:21:42.631 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:21:42.637 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA2Go5pNonJJRAATIScnKWvhrtk3CRroiw/uw== 2026-02-12T18:21:42.637 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA2Go5pNonJJRAATIScnKWvhrtk3CRroiw/uw=="}' 2026-02-12T18:21:42.637 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 956db082-779b-4d45-89ba-86e1eea4808c -i td/test-erasure-eio/3/new.json 2026-02-12T18:21:42.867 INFO:tasks.workunit.client.0.trial116.stdout:3 2026-02-12T18:21:42.874 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-12T18:21:42.875 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA2Go5pNonJJRAATIScnKWvhrtk3CRroiw/uw== --osd-uuid 956db082-779b-4d45-89ba-86e1eea4808c 2026-02-12T18:21:42.890 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:42.887+0000 7fed6d030780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:42.891 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:42.889+0000 7fed6d030780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:42.892 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:42.889+0000 7fed6d030780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:42.892 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:42.890+0000 7fed6d030780 -1 bdev(0x3a0ec00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-12T18:21:42.893 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:42.890+0000 7fed6d030780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-12T18:21:43.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-12T18:21:43.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:21:43.216 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-12T18:21:43.216 INFO:tasks.workunit.client.0.trial116.stdout:adding osd3 key to auth repository 2026-02-12T18:21:43.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:21:43.531 INFO:tasks.workunit.client.0.trial116.stdout:start osd.3 2026-02-12T18:21:43.532 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-12T18:21:43.532 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:21:43.533 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:21:43.533 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:21:43.533 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:21:43.546 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:43.542+0000 7ff45c62f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:43.547 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:43.544+0000 7ff45c62f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:43.548 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:43.545+0000 7ff45c62f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:43.684 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:43.681+0000 7ff45c62f780 -1 Falling back to public interface 2026-02-12T18:21:43.756 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-12T18:21:43.756 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:21:43.757 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-12T18:21:43.757 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:21:43.757 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:21:43.757 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:21:43.757 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:21:43.757 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:21:43.758 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:21:43.758 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:21:43.987 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:21:43.991 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:43.987+0000 7ff45c62f780 -1 osd.3 0 log_to_monitors true 2026-02-12T18:21:44.989 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:21:44.989 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:21:44.990 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:21:44.990 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:21:44.990 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:21:44.990 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:21:45.222 INFO:tasks.workunit.client.0.trial116.stdout:osd.3 up in weight 1 up_from 27 up_thru 28 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2346278200,v1:127.0.0.1:6827/2346278200] [v2:127.0.0.1:6828/2346278200,v1:127.0.0.1:6829/2346278200] exists,up 956db082-779b-4d45-89ba-86e1eea4808c 2026-02-12T18:21:45.226 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:21:45.226 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:21:45.226 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:21:45.227 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:21:45.227 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-12T18:21:45.227 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:21:45.227 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:21:45.227 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-12T18:21:45.227 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:21:45.227 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-12T18:21:45.227 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:21:45.227 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:21:45.227 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:21:45.227 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:21:45.228 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-12T18:21:45.228 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-12T18:21:45.228 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:21:45.228 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:21:45.228 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:21:45.228 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:21:45.228 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:21:45.228 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:21:45.228 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:21:45.228 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:21:45.229 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:21:45.229 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:21:45.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:21:45.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:21:45.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:21:45.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:21:45.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:21:45.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:21:45.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:21:45.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:21:45.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:21:45.230 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:21:45.230 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:21:45.230 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:21:45.230 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-12T18:21:45.230 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:21:45.230 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f25357e6-cf0f-4471-8f53-08fa2a25c4c0 2026-02-12T18:21:45.230 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 f25357e6-cf0f-4471-8f53-08fa2a25c4c0' 2026-02-12T18:21:45.230 INFO:tasks.workunit.client.0.trial116.stdout:add osd4 f25357e6-cf0f-4471-8f53-08fa2a25c4c0 2026-02-12T18:21:45.231 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:21:45.236 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA5Go5ppeLjDRAAErJMmEgk2xH3sogPCQbcdQ== 2026-02-12T18:21:45.236 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA5Go5ppeLjDRAAErJMmEgk2xH3sogPCQbcdQ=="}' 2026-02-12T18:21:45.236 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f25357e6-cf0f-4471-8f53-08fa2a25c4c0 -i td/test-erasure-eio/4/new.json 2026-02-12T18:21:45.464 INFO:tasks.workunit.client.0.trial116.stdout:4 2026-02-12T18:21:45.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-12T18:21:45.473 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA5Go5ppeLjDRAAErJMmEgk2xH3sogPCQbcdQ== --osd-uuid f25357e6-cf0f-4471-8f53-08fa2a25c4c0 2026-02-12T18:21:45.489 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:45.485+0000 7f40bbc0e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:45.490 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:45.487+0000 7f40bbc0e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:45.491 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:45.487+0000 7f40bbc0e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:45.492 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:45.488+0000 7f40bbc0e780 -1 bdev(0x38c6c00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-12T18:21:45.492 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:45.488+0000 7f40bbc0e780 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-12T18:21:45.818 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-12T18:21:45.818 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:21:45.819 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-12T18:21:45.819 INFO:tasks.workunit.client.0.trial116.stdout:adding osd4 key to auth repository 2026-02-12T18:21:45.819 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:21:46.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-12T18:21:46.135 INFO:tasks.workunit.client.0.trial116.stdout:start osd.4 2026-02-12T18:21:46.136 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:21:46.136 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:21:46.136 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:21:46.136 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:21:46.150 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:46.147+0000 7ff448e32780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:46.152 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:46.148+0000 7ff448e32780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:46.153 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:46.149+0000 7ff448e32780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:46.285 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:46.281+0000 7ff448e32780 -1 Falling back to public interface 2026-02-12T18:21:46.371 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-12T18:21:46.371 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:21:46.371 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-12T18:21:46.371 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:21:46.371 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:21:46.371 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:21:46.371 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:21:46.372 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:21:46.372 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:21:46.372 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-12T18:21:46.595 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:21:46.627 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:46.623+0000 7ff448e32780 -1 osd.4 0 log_to_monitors true 2026-02-12T18:21:47.597 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:21:47.597 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:21:47.597 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:21:47.598 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:21:47.598 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:21:47.598 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-12T18:21:47.828 INFO:tasks.workunit.client.0.trial116.stdout:osd.4 up in weight 1 up_from 35 up_thru 36 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/1559613344,v1:127.0.0.1:6835/1559613344] [v2:127.0.0.1:6836/1559613344,v1:127.0.0.1:6837/1559613344] exists,up f25357e6-cf0f-4471-8f53-08fa2a25c4c0 2026-02-12T18:21:47.832 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:21:47.832 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:21:47.832 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:21:47.832 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:21:47.832 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-12T18:21:47.832 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:21:47.832 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:21:47.832 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-12T18:21:47.832 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:21:47.833 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-12T18:21:47.833 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:21:47.833 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:21:47.833 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:21:47.833 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:21:47.833 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-12T18:21:47.833 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-12T18:21:47.833 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:21:47.833 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:21:47.833 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:21:47.833 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:21:47.833 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:21:47.834 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:21:47.834 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:21:47.834 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:21:47.834 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:21:47.834 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:21:47.834 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:21:47.834 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:21:47.834 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:21:47.834 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:21:47.834 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:21:47.834 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:21:47.834 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:21:47.834 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:21:47.835 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:21:47.835 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:21:47.835 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:21:47.835 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:21:47.835 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-12T18:21:47.835 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:21:47.835 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=eced2abd-b52b-4538-8ade-2ff0eb78e2a9 2026-02-12T18:21:47.835 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 eced2abd-b52b-4538-8ade-2ff0eb78e2a9' 2026-02-12T18:21:47.836 INFO:tasks.workunit.client.0.trial116.stdout:add osd5 eced2abd-b52b-4538-8ade-2ff0eb78e2a9 2026-02-12T18:21:47.836 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:21:47.842 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA7Go5pRfT+MRAAmxZb0ebnU+o7NQorI0Wq0g== 2026-02-12T18:21:47.842 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA7Go5pRfT+MRAAmxZb0ebnU+o7NQorI0Wq0g=="}' 2026-02-12T18:21:47.842 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new eced2abd-b52b-4538-8ade-2ff0eb78e2a9 -i td/test-erasure-eio/5/new.json 2026-02-12T18:21:48.073 INFO:tasks.workunit.client.0.trial116.stdout:5 2026-02-12T18:21:48.080 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-12T18:21:48.081 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA7Go5pRfT+MRAAmxZb0ebnU+o7NQorI0Wq0g== --osd-uuid eced2abd-b52b-4538-8ade-2ff0eb78e2a9 2026-02-12T18:21:48.096 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:48.093+0000 7f70f3471780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:48.098 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:48.094+0000 7f70f3471780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:48.099 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:48.095+0000 7f70f3471780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:48.099 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:48.095+0000 7f70f3471780 -1 bdev(0x3754c00 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-12T18:21:48.099 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:48.095+0000 7f70f3471780 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-12T18:21:48.410 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-12T18:21:48.411 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:21:48.411 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-12T18:21:48.412 INFO:tasks.workunit.client.0.trial116.stdout:adding osd5 key to auth repository 2026-02-12T18:21:48.412 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:21:48.738 INFO:tasks.workunit.client.0.trial116.stdout:start osd.5 2026-02-12T18:21:48.739 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-12T18:21:48.739 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:21:48.739 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:21:48.739 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:21:48.739 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:21:48.753 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:48.749+0000 7f822140f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:48.754 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:48.751+0000 7f822140f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:48.755 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:48.751+0000 7f822140f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:48.928 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:48.925+0000 7f822140f780 -1 Falling back to public interface 2026-02-12T18:21:48.965 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-12T18:21:48.966 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:21:48.966 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-12T18:21:48.966 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:21:48.966 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:21:48.966 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:21:48.966 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:21:48.966 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:21:48.967 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:21:48.967 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-12T18:21:49.191 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:21:49.245 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:49.241+0000 7f822140f780 -1 osd.5 0 log_to_monitors true 2026-02-12T18:21:50.192 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:21:50.192 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:21:50.193 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:21:50.193 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:21:50.193 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:21:50.194 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-12T18:21:50.415 INFO:tasks.workunit.client.0.trial116.stdout:osd.5 up in weight 1 up_from 43 up_thru 44 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/3577115173,v1:127.0.0.1:6843/3577115173] [v2:127.0.0.1:6844/3577115173,v1:127.0.0.1:6845/3577115173] exists,up eced2abd-b52b-4538-8ade-2ff0eb78e2a9 2026-02-12T18:21:50.416 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:21:50.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:21:50.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:21:50.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:21:50.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-12T18:21:50.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:21:50.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:21:50.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-12T18:21:50.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:21:50.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-12T18:21:50.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:21:50.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:21:50.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:21:50.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:21:50.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-12T18:21:50.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-12T18:21:50.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:21:50.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:21:50.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:21:50.417 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:21:50.417 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:21:50.417 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:21:50.418 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:21:50.418 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:21:50.418 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:21:50.418 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:21:50.418 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:21:50.418 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:21:50.418 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:21:50.418 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:21:50.418 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:21:50.418 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:21:50.418 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:21:50.418 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:21:50.418 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:21:50.418 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:21:50.418 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:21:50.418 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:21:50.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-12T18:21:50.419 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:21:50.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a3c07f43-6c0f-472a-a1a1-fc3d94e2485c 2026-02-12T18:21:50.420 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 a3c07f43-6c0f-472a-a1a1-fc3d94e2485c' 2026-02-12T18:21:50.420 INFO:tasks.workunit.client.0.trial116.stdout:add osd6 a3c07f43-6c0f-472a-a1a1-fc3d94e2485c 2026-02-12T18:21:50.420 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:21:50.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA+Go5pp790GRAASpVkHknl55sd56QG/CJQ7w== 2026-02-12T18:21:50.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA+Go5pp790GRAASpVkHknl55sd56QG/CJQ7w=="}' 2026-02-12T18:21:50.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a3c07f43-6c0f-472a-a1a1-fc3d94e2485c -i td/test-erasure-eio/6/new.json 2026-02-12T18:21:50.668 INFO:tasks.workunit.client.0.trial116.stdout:6 2026-02-12T18:21:50.676 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-12T18:21:50.677 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA+Go5pp790GRAASpVkHknl55sd56QG/CJQ7w== --osd-uuid a3c07f43-6c0f-472a-a1a1-fc3d94e2485c 2026-02-12T18:21:50.692 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:50.688+0000 7f9e5083f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:50.693 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:50.690+0000 7f9e5083f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:50.694 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:50.690+0000 7f9e5083f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:50.695 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:50.691+0000 7f9e5083f780 -1 bdev(0x4146c00 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-12T18:21:50.695 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:50.691+0000 7f9e5083f780 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-12T18:21:51.031 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-12T18:21:51.032 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:21:51.032 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-12T18:21:51.032 INFO:tasks.workunit.client.0.trial116.stdout:adding osd6 key to auth repository 2026-02-12T18:21:51.033 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:21:51.354 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-12T18:21:51.355 INFO:tasks.workunit.client.0.trial116.stdout:start osd.6 2026-02-12T18:21:51.356 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:21:51.356 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:21:51.356 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:21:51.356 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:21:51.370 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:51.366+0000 7f5c44033780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:51.371 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:51.367+0000 7f5c44033780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:51.372 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:51.368+0000 7f5c44033780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:21:51.552 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:51.548+0000 7f5c44033780 -1 Falling back to public interface 2026-02-12T18:21:51.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-12T18:21:51.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:21:51.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-12T18:21:51.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:21:51.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:21:51.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:21:51.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:21:51.586 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:21:51.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:21:51.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-12T18:21:51.813 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:21:51.899 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:21:51.896+0000 7f5c44033780 -1 osd.6 0 log_to_monitors true 2026-02-12T18:21:52.814 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:21:52.815 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:21:52.815 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:21:52.815 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:21:52.816 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:21:52.816 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-12T18:21:53.037 INFO:tasks.workunit.client.0.trial116.stdout:osd.6 up in weight 1 up_from 50 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/74857984,v1:127.0.0.1:6851/74857984] [v2:127.0.0.1:6852/74857984,v1:127.0.0.1:6853/74857984] exists,up a3c07f43-6c0f-472a-a1a1-fc3d94e2485c 2026-02-12T18:21:53.038 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:21:53.038 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:21:53.038 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:21:53.038 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-12T18:21:53.038 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T18:21:53.039 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T18:21:53.039 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:21:53.039 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:21:53.039 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:21:53.039 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.0.asok 2026-02-12T18:21:53.039 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-12T18:21:53.040 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-osd.0.asok log flush 2026-02-12T18:21:53.084 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:21:53.089 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-12T18:21:53.091 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:21:36.188+0000 7fa8aa043780 0 load: jerasure load: lrc 2026-02-12T18:21:53.092 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:415: TEST_ec_single_recovery_error: local poolname=pool-jerasure 2026-02-12T18:21:53.092 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:416: TEST_ec_single_recovery_error: create_erasure_coded_pool pool-jerasure 3 2 2026-02-12T18:21:53.092 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-12T18:21:53.093 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-12T18:21:53.093 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-12T18:21:53.093 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-12T18:21:53.093 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-12T18:21:53.093 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-12T18:21:53.093 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-12T18:21:53.378 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-12T18:21:53.378 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-12T18:21:53.755 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-jerasure' already exists 2026-02-12T18:21:53.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:21:54.764 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-12T18:21:54.765 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:21:54.765 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:21:54.765 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:21:54.765 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:21:54.766 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:21:54.766 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:21:54.766 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:21:54.766 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:21:54.766 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:21:54.853 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:21:54.853 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:21:54.853 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:21:54.853 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:21:54.853 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:21:54.854 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:21:55.082 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:21:55.082 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:21:55.082 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:21:55.082 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:21:55.082 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:21:55.082 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:21:55.082 INFO:tasks.workunit.client.0.trial116.stderr:6' 2026-02-12T18:21:55.082 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:21:55.082 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:21:55.082 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:21:55.152 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803782 2026-02-12T18:21:55.152 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803782 2026-02-12T18:21:55.152 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782' 2026-02-12T18:21:55.152 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:21:55.152 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:21:55.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574854 2026-02-12T18:21:55.216 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574854 2026-02-12T18:21:55.216 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574854' 2026-02-12T18:21:55.216 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:21:55.216 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:21:55.279 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411333 2026-02-12T18:21:55.279 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411333 2026-02-12T18:21:55.279 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-77309411333' 2026-02-12T18:21:55.279 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:21:55.279 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:21:55.344 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116997 2026-02-12T18:21:55.344 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116997 2026-02-12T18:21:55.344 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-77309411333 3-115964116997' 2026-02-12T18:21:55.344 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:21:55.344 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-12T18:21:55.406 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855364 2026-02-12T18:21:55.406 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855364 2026-02-12T18:21:55.407 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-77309411333 3-115964116997 4-150323855364' 2026-02-12T18:21:55.407 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:21:55.407 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-12T18:21:55.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593732 2026-02-12T18:21:55.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593732 2026-02-12T18:21:55.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-77309411333 3-115964116997 4-150323855364 5-184683593732' 2026-02-12T18:21:55.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:21:55.473 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-12T18:21:55.536 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=214748364803 2026-02-12T18:21:55.536 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 214748364803 2026-02-12T18:21:55.536 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-77309411333 3-115964116997 4-150323855364 5-184683593732 6-214748364803' 2026-02-12T18:21:55.536 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:21:55.537 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803782 2026-02-12T18:21:55.537 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:21:55.537 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:21:55.538 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803782 2026-02-12T18:21:55.538 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:21:55.539 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803782 2026-02-12T18:21:55.539 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803782' 2026-02-12T18:21:55.539 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.0 seq 25769803782 2026-02-12T18:21:55.540 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:21:55.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803780 -lt 25769803782 2026-02-12T18:21:55.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:21:56.769 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:21:56.770 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:21:56.997 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803782 2026-02-12T18:21:56.997 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:21:56.997 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574854 2026-02-12T18:21:56.998 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:21:56.998 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:21:56.999 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574854 2026-02-12T18:21:56.999 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:21:57.000 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574854 2026-02-12T18:21:57.000 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574854' 2026-02-12T18:21:57.000 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.1 seq 55834574854 2026-02-12T18:21:57.001 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:21:57.228 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574854 -lt 55834574854 2026-02-12T18:21:57.228 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:21:57.229 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411333 2026-02-12T18:21:57.229 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:21:57.230 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:21:57.230 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411333 2026-02-12T18:21:57.231 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:21:57.232 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411333 2026-02-12T18:21:57.232 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411333' 2026-02-12T18:21:57.232 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.2 seq 77309411333 2026-02-12T18:21:57.232 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:21:57.463 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411333 -lt 77309411333 2026-02-12T18:21:57.463 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:21:57.463 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-115964116997 2026-02-12T18:21:57.464 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:21:57.464 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:21:57.465 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964116997 2026-02-12T18:21:57.465 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:21:57.466 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964116997 2026-02-12T18:21:57.466 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 115964116997' 2026-02-12T18:21:57.467 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.3 seq 115964116997 2026-02-12T18:21:57.467 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:21:57.692 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964116997 -lt 115964116997 2026-02-12T18:21:57.693 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:21:57.693 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-150323855364 2026-02-12T18:21:57.693 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:21:57.694 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-12T18:21:57.695 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-150323855364 2026-02-12T18:21:57.695 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:21:57.696 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=150323855364 2026-02-12T18:21:57.696 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 150323855364' 2026-02-12T18:21:57.696 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.4 seq 150323855364 2026-02-12T18:21:57.697 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-12T18:21:57.922 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 150323855364 -lt 150323855364 2026-02-12T18:21:57.922 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:21:57.923 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-184683593732 2026-02-12T18:21:57.923 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:21:57.923 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-12T18:21:57.924 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-184683593732 2026-02-12T18:21:57.924 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:21:57.925 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593732 2026-02-12T18:21:57.925 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 184683593732' 2026-02-12T18:21:57.926 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.5 seq 184683593732 2026-02-12T18:21:57.926 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-12T18:21:58.152 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593732 -lt 184683593732 2026-02-12T18:21:58.152 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:21:58.153 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 6-214748364803 2026-02-12T18:21:58.153 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:21:58.153 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=6 2026-02-12T18:21:58.154 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-214748364803 2026-02-12T18:21:58.154 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:21:58.155 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=214748364803 2026-02-12T18:21:58.155 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.6 seq 214748364803' 2026-02-12T18:21:58.156 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.6 seq 214748364803 2026-02-12T18:21:58.156 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-12T18:21:58.386 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 214748364804 -lt 214748364803 2026-02-12T18:21:58.387 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:21:58.387 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:21:58.387 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:21:58.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:21:58.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:21:58.709 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:21:58.709 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:21:58.709 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:21:58.709 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:21:58.709 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:21:58.710 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:21:58.935 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:21:58.936 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:21:58.936 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:21:58.936 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:21:59.253 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:21:59.253 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:21:59.253 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:21:59.253 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:418: TEST_ec_single_recovery_error: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-12T18:21:59.253 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-12T18:21:59.253 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-12T18:21:59.253 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-12T18:21:59.253 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:21:59.253 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-12T18:21:59.253 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:21:59.254 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-12T18:21:59.254 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:21:59.254 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-12T18:21:59.254 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:21:59.254 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-12T18:21:59.254 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-12T18:21:59.270 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:419: TEST_ec_single_recovery_error: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 0 2026-02-12T18:21:59.270 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-12T18:21:59.270 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-12T18:21:59.270 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-12T18:21:59.270 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-12T18:21:59.270 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-12T18:21:59.271 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-12T18:21:59.271 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=myobject 2026-02-12T18:21:59.271 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-12T18:21:59.271 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-12T18:21:59.271 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-12T18:21:59.271 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=0 2026-02-12T18:21:59.271 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-12T18:21:59.271 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure myobject 2026-02-12T18:21:59.271 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:21:59.271 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-12T18:21:59.271 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-12T18:21:59.271 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:21:59.501 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:21:59.502 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:21:59.502 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:21:59.502 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:21:59.502 INFO:tasks.workunit.client.0.trial116.stderr:6' 2026-02-12T18:21:59.502 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2026-02-12T18:21:59.502 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '5' '1' '0' '6') 2026-02-12T18:21:59.502 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-12T18:21:59.502 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=3 2026-02-12T18:21:59.502 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-12T18:21:59.502 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/3/type 2026-02-12T18:21:59.504 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-12T18:21:59.504 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-12T18:21:59.504 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-12T18:21:59.505 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-12T18:21:59.505 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-12T18:21:59.505 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-12T18:21:59.505 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-12T18:21:59.505 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-12T18:21:59.505 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-12T18:21:59.505 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-12T18:21:59.505 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:21:59.505 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:21:59.505 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:21:59.505 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.3.asok 2026-02-12T18:21:59.505 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.76821/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-12T18:21:59.551 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-12T18:21:59.551 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-12T18:21:59.552 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-12T18:21:59.553 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.3 2026-02-12T18:21:59.553 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-12T18:21:59.553 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-12T18:21:59.553 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:21:59.553 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:21:59.554 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:21:59.554 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.3.asok 2026-02-12T18:21:59.554 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-12T18:21:59.554 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-osd.3.asok injectdataerr pool-jerasure myobject 0 2026-02-12T18:21:59.600 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:421: TEST_ec_single_recovery_error: get_osds pool-jerasure myobject 2026-02-12T18:21:59.600 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:21:59.600 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-12T18:21:59.601 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:21:59.601 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-12T18:21:59.826 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:21:59.827 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:21:59.827 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:21:59.827 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:21:59.827 INFO:tasks.workunit.client.0.trial116.stderr:6' 2026-02-12T18:21:59.827 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2026-02-12T18:21:59.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:421: TEST_ec_single_recovery_error: initial_osds=('3' '5' '1' '0' '6') 2026-02-12T18:21:59.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:421: TEST_ec_single_recovery_error: local -a initial_osds 2026-02-12T18:21:59.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:422: TEST_ec_single_recovery_error: local last_osd=6 2026-02-12T18:21:59.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:424: TEST_ec_single_recovery_error: kill_daemons td/test-erasure-eio TERM osd.6 2026-02-12T18:21:59.827 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:21:59.827 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:21:59.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:21:59.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:21:59.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:22:00.133 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:22:00.133 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:425: TEST_ec_single_recovery_error: ceph osd down 6 2026-02-12T18:22:00.358 INFO:tasks.workunit.client.0.trial116.stderr:osd.6 is already down. 2026-02-12T18:22:00.365 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:426: TEST_ec_single_recovery_error: ceph osd out 6 2026-02-12T18:22:00.646 INFO:tasks.workunit.client.0.trial116.stderr:osd.6 is already out. 2026-02-12T18:22:00.655 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:429: TEST_ec_single_recovery_error: wait_for_clean 2026-02-12T18:22:00.655 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:22:00.655 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:22:00.655 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:22:00.656 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:22:00.656 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:22:00.656 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:22:00.656 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:22:00.656 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:22:00.657 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:22:00.744 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:22:00.745 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:22:00.745 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:22:00.745 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:22:00.745 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:22:00.745 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:22:00.969 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:22:00.969 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:22:00.969 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:22:00.969 INFO:tasks.workunit.client.0.trial116.stderr:3 2026-02-12T18:22:00.969 INFO:tasks.workunit.client.0.trial116.stderr:4 2026-02-12T18:22:00.969 INFO:tasks.workunit.client.0.trial116.stderr:5 2026-02-12T18:22:00.969 INFO:tasks.workunit.client.0.trial116.stderr:6' 2026-02-12T18:22:00.970 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:22:00.970 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:00.970 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:22:01.036 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803785 2026-02-12T18:22:01.036 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803785 2026-02-12T18:22:01.036 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785' 2026-02-12T18:22:01.036 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:01.036 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:22:01.100 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574857 2026-02-12T18:22:01.100 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574857 2026-02-12T18:22:01.100 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574857' 2026-02-12T18:22:01.100 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:01.100 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:22:01.165 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411336 2026-02-12T18:22:01.165 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411336 2026-02-12T18:22:01.165 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-77309411336' 2026-02-12T18:22:01.165 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:01.165 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:22:01.231 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964117000 2026-02-12T18:22:01.231 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964117000 2026-02-12T18:22:01.231 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-77309411336 3-115964117000' 2026-02-12T18:22:01.231 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:01.231 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-12T18:22:01.295 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855367 2026-02-12T18:22:01.295 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855367 2026-02-12T18:22:01.295 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-77309411336 3-115964117000 4-150323855367' 2026-02-12T18:22:01.295 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:01.295 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-12T18:22:01.361 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593735 2026-02-12T18:22:01.362 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593735 2026-02-12T18:22:01.362 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-77309411336 3-115964117000 4-150323855367 5-184683593735' 2026-02-12T18:22:01.362 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:01.362 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-12T18:22:01.407 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-12T18:22:01.407 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: true 2026-02-12T18:22:01.407 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-12T18:22:01.407 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: continue 2026-02-12T18:22:01.407 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:22:01.408 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803785 2026-02-12T18:22:01.408 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:22:01.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:22:01.409 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803785 2026-02-12T18:22:01.409 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:22:01.410 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803785 2026-02-12T18:22:01.410 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803785' 2026-02-12T18:22:01.410 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.0 seq 25769803785 2026-02-12T18:22:01.410 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:22:01.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803783 -lt 25769803785 2026-02-12T18:22:01.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:22:02.637 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:22:02.638 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:22:02.864 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803785 -lt 25769803785 2026-02-12T18:22:02.865 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:22:02.865 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574857 2026-02-12T18:22:02.865 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:22:02.866 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:22:02.866 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574857 2026-02-12T18:22:02.867 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:22:02.868 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574857 2026-02-12T18:22:02.868 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574857' 2026-02-12T18:22:02.868 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.1 seq 55834574857 2026-02-12T18:22:02.868 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:22:03.096 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574857 -lt 55834574857 2026-02-12T18:22:03.096 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:22:03.097 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411336 2026-02-12T18:22:03.097 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:22:03.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:22:03.098 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411336 2026-02-12T18:22:03.098 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:22:03.099 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411336 2026-02-12T18:22:03.100 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411336' 2026-02-12T18:22:03.100 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.2 seq 77309411336 2026-02-12T18:22:03.100 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:22:03.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411336 -lt 77309411336 2026-02-12T18:22:03.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:22:03.327 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-115964117000 2026-02-12T18:22:03.327 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:22:03.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:22:03.328 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964117000 2026-02-12T18:22:03.329 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:22:03.330 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964117000 2026-02-12T18:22:03.330 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 115964117000' 2026-02-12T18:22:03.330 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.3 seq 115964117000 2026-02-12T18:22:03.331 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:22:03.558 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964117000 -lt 115964117000 2026-02-12T18:22:03.558 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:22:03.559 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 4-150323855367 2026-02-12T18:22:03.559 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:22:03.559 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=4 2026-02-12T18:22:03.560 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-150323855367 2026-02-12T18:22:03.560 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:22:03.561 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=150323855367 2026-02-12T18:22:03.561 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.4 seq 150323855367' 2026-02-12T18:22:03.562 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.4 seq 150323855367 2026-02-12T18:22:03.562 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-12T18:22:03.789 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 150323855367 -lt 150323855367 2026-02-12T18:22:03.789 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:22:03.790 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 5-184683593735 2026-02-12T18:22:03.790 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:22:03.791 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=5 2026-02-12T18:22:03.791 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-184683593735 2026-02-12T18:22:03.791 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:22:03.792 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593735 2026-02-12T18:22:03.792 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.5 seq 184683593735' 2026-02-12T18:22:03.793 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.5 seq 184683593735 2026-02-12T18:22:03.793 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-12T18:22:04.022 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593735 -lt 184683593735 2026-02-12T18:22:04.023 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:22:04.023 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:22:04.023 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:22:04.341 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:22:04.341 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:22:04.341 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:22:04.341 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:22:04.341 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:22:04.342 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:22:04.342 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:22:04.342 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:22:04.562 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:22:04.563 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:22:04.563 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:22:04.563 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:22:04.876 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:22:04.876 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:22:04.876 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:22:04.876 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:431: TEST_ec_single_recovery_error: rados_get td/test-erasure-eio pool-jerasure myobject 2026-02-12T18:22:04.876 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-12T18:22:04.876 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-12T18:22:04.876 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=myobject 2026-02-12T18:22:04.876 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-12T18:22:04.876 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-12T18:22:04.876 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get myobject td/test-erasure-eio/COPY 2026-02-12T18:22:04.894 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-12T18:22:04.895 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-12T18:22:04.896 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:433: TEST_ec_single_recovery_error: delete_erasure_coded_pool pool-jerasure 2026-02-12T18:22:04.896 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-12T18:22:04.896 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-12T18:22:05.181 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-jerasure' does not exist 2026-02-12T18:22:05.190 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-12T18:22:05.470 INFO:tasks.workunit.client.0.trial116.stderr:erasure-code-profile myprofile does not exist 2026-02-12T18:22:05.478 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-12T18:22:05.479 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-12T18:22:05.479 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:22:05.479 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-12T18:22:05.480 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:22:05.480 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:22:05.480 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:22:05.480 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:22:05.480 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:22:05.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:22:05.590 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:22:05.591 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:22:05.591 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:22:05.592 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:22:05.592 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:22:05.593 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:22:05.593 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:22:05.594 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:22:05.594 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:22:05.594 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:22:05.595 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:22:05.596 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:22:05.596 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-12T18:22:05.611 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:22:05.611 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:22:05.611 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:22:05.611 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76821 2026-02-12T18:22:05.613 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:22:05.613 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:22:05.613 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-12T18:22:05.613 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-12T18:22:05.613 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-12T18:22:05.613 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-12T18:22:05.613 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-12T18:22:05.614 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:22:05.614 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-12T18:22:05.614 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:22:05.614 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:22:05.614 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:22:05.614 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:22:05.614 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:22:05.615 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:22:05.615 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:22:05.615 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:22:05.616 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:22:05.617 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:22:05.617 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:22:05.617 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:22:05.618 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:22:05.618 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:22:05.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:22:05.619 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:22:05.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:22:05.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:22:05.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-12T18:22:05.621 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:22:05.622 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:22:05.622 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:22:05.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76821 2026-02-12T18:22:05.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:22:05.623 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:22:05.623 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-12T18:22:05.624 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T18:22:05.624 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:22:05.624 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:22:05.624 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76821 2026-02-12T18:22:05.625 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T18:22:05.626 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T18:22:05.626 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T18:22:05.626 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-12T18:22:05.626 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-12T18:22:05.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-12T18:22:05.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T18:22:05.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T18:22:05.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T18:22:05.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-12T18:22:05.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-12T18:22:05.644 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T18:22:05.644 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:22:05.644 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:22:05.644 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:22:05.644 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:22:05.644 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:22:05.644 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:22:05.644 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T18:22:05.667 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T18:22:05.669 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T18:22:05.669 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:22:05.669 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:22:05.669 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T18:22:05.669 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T18:22:05.670 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:22:05.670 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:22:05.670 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:22:05.671 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:22:05.671 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:22:05.671 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:22:05.671 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:22:05.671 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:22:05.671 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76821/ceph-mon.a.asok config get fsid 2026-02-12T18:22:05.713 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T18:22:05.713 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:22:05.713 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:22:05.713 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T18:22:05.714 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T18:22:05.714 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:22:05.714 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:22:05.714 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:22:05.714 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:22:05.714 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:22:05.714 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:22:05.714 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:22:05.715 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:22:05.715 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76821/ceph-mon.a.asok config get mon_host 2026-02-12T18:22:05.756 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-12T18:22:05.756 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-12T18:22:05.756 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T18:22:05.756 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T18:22:05.756 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T18:22:05.756 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-12T18:22:05.756 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T18:22:05.855 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T18:22:05.855 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:22:05.855 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:22:05.855 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:22:05.856 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:22:05.856 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:22:05.856 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:22:05.856 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:22:05.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:22:05.872 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-12T18:22:05.872 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T18:22:05.966 INFO:tasks.workunit.client.0.trial116.stderr:pool 'rbd' already exists 2026-02-12T18:22:05.972 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:22:06.975 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-12T18:22:06.975 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:22:06.975 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:22:06.975 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:22:06.975 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:22:06.975 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:22:06.976 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:22:06.976 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-12T18:22:06.976 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-mon.a.asok log flush 2026-02-12T18:22:07.015 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:22:07.020 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-12T18:22:07.021 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:22:05.657+0000 7fe99343cc00 0 load: jerasure load: lrc 2026-02-12T18:22:07.022 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_bad_size_shard_0 td/test-erasure-eio 2026-02-12T18:22:07.022 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:322: TEST_rados_get_bad_size_shard_0: local dir=td/test-erasure-eio 2026-02-12T18:22:07.022 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:323: TEST_rados_get_bad_size_shard_0: setup_osds 4 2026-02-12T18:22:07.022 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-12T18:22:07.022 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-12T18:22:07.023 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-12T18:22:07.023 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-12T18:22:07.026 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:22:07.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-12T18:22:07.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:22:07.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:22:07.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T18:22:07.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:22:07.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-12T18:22:07.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:22:07.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:22:07.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:22:07.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:22:07.028 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-12T18:22:07.028 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-12T18:22:07.028 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:22:07.028 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:22:07.028 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:22:07.028 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:22:07.028 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:22:07.028 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:22:07.028 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:22:07.028 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:22:07.028 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:22:07.028 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:22:07.028 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:22:07.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:22:07.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:22:07.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:22:07.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:22:07.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:22:07.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:22:07.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:22:07.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:22:07.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:22:07.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:22:07.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:22:07.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-12T18:22:07.030 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:22:07.030 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1d0a5514-6bc8-4423-9b00-bb5542ad8289 2026-02-12T18:22:07.030 INFO:tasks.workunit.client.0.trial116.stdout:add osd0 1d0a5514-6bc8-4423-9b00-bb5542ad8289 2026-02-12T18:22:07.030 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 1d0a5514-6bc8-4423-9b00-bb5542ad8289' 2026-02-12T18:22:07.031 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:22:07.037 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBPGo5pgeYDAhAAUC5iSSotmcOyR5W3fCILiQ== 2026-02-12T18:22:07.037 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBPGo5pgeYDAhAAUC5iSSotmcOyR5W3fCILiQ=="}' 2026-02-12T18:22:07.037 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1d0a5514-6bc8-4423-9b00-bb5542ad8289 -i td/test-erasure-eio/0/new.json 2026-02-12T18:22:07.138 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:22:07.145 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-12T18:22:07.146 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBPGo5pgeYDAhAAUC5iSSotmcOyR5W3fCILiQ== --osd-uuid 1d0a5514-6bc8-4423-9b00-bb5542ad8289 2026-02-12T18:22:07.161 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:07.158+0000 7f6a6d231780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:07.163 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:07.159+0000 7f6a6d231780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:07.164 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:07.160+0000 7f6a6d231780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:07.164 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:07.160+0000 7f6a6d231780 -1 bdev(0x2d2cc00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-12T18:22:07.164 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:07.160+0000 7f6a6d231780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-12T18:22:07.615 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-12T18:22:07.615 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:22:07.615 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T18:22:07.616 INFO:tasks.workunit.client.0.trial116.stdout:adding osd0 key to auth repository 2026-02-12T18:22:07.616 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:22:07.720 INFO:tasks.workunit.client.0.trial116.stdout:start osd.0 2026-02-12T18:22:07.721 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T18:22:07.721 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:22:07.721 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:22:07.722 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:22:07.722 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:22:07.735 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:07.731+0000 7f58a8e29780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:07.736 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:07.733+0000 7f58a8e29780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:07.737 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:07.734+0000 7f58a8e29780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:07.817 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T18:22:07.817 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:22:07.817 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T18:22:07.817 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:22:07.817 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:22:07.817 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:22:07.818 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:22:07.818 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:22:07.818 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:22:07.818 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:22:07.914 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:22:07.928 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:07.925+0000 7f58a8e29780 -1 Falling back to public interface 2026-02-12T18:22:08.234 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:08.230+0000 7f58a8e29780 -1 osd.0 0 log_to_monitors true 2026-02-12T18:22:08.915 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:22:08.915 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:22:08.916 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:22:08.916 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:22:08.917 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:22:08.917 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:22:09.141 INFO:tasks.workunit.client.0.trial116.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/640021804,v1:127.0.0.1:6803/640021804] [v2:127.0.0.1:6804/640021804,v1:127.0.0.1:6805/640021804] exists,up 1d0a5514-6bc8-4423-9b00-bb5542ad8289 2026-02-12T18:22:09.145 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:22:09.145 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:22:09.145 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:22:09.145 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:22:09.145 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-12T18:22:09.145 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:22:09.145 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:22:09.145 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-12T18:22:09.146 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:22:09.146 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-12T18:22:09.146 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:22:09.146 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:22:09.146 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:22:09.146 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:22:09.146 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-12T18:22:09.146 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-12T18:22:09.146 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:22:09.146 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:22:09.146 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:22:09.146 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:22:09.147 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:22:09.147 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:22:09.147 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:22:09.147 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:22:09.147 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:22:09.147 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:22:09.147 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:22:09.147 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:22:09.147 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:22:09.147 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:22:09.147 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:22:09.147 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:22:09.147 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:22:09.148 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:22:09.148 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:22:09.148 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:22:09.148 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:22:09.148 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:22:09.148 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-12T18:22:09.148 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:22:09.149 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b2d08ec4-55c3-47fb-a37a-56257fef04aa 2026-02-12T18:22:09.149 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 b2d08ec4-55c3-47fb-a37a-56257fef04aa' 2026-02-12T18:22:09.149 INFO:tasks.workunit.client.0.trial116.stdout:add osd1 b2d08ec4-55c3-47fb-a37a-56257fef04aa 2026-02-12T18:22:09.149 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:22:09.155 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBRGo5pbewPCRAAOkwONSs0IuTKLDlOUHbDlg== 2026-02-12T18:22:09.155 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBRGo5pbewPCRAAOkwONSs0IuTKLDlOUHbDlg=="}' 2026-02-12T18:22:09.155 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b2d08ec4-55c3-47fb-a37a-56257fef04aa -i td/test-erasure-eio/1/new.json 2026-02-12T18:22:09.384 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:22:09.392 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-12T18:22:09.393 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBRGo5pbewPCRAAOkwONSs0IuTKLDlOUHbDlg== --osd-uuid b2d08ec4-55c3-47fb-a37a-56257fef04aa 2026-02-12T18:22:09.410 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:09.406+0000 7f20e4a27780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:09.411 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:09.408+0000 7f20e4a27780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:09.412 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:09.408+0000 7f20e4a27780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:09.413 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:09.409+0000 7f20e4a27780 -1 bdev(0x353ec00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-12T18:22:09.413 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:09.409+0000 7f20e4a27780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-12T18:22:09.717 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-12T18:22:09.717 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:22:09.718 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-12T18:22:09.718 INFO:tasks.workunit.client.0.trial116.stdout:adding osd1 key to auth repository 2026-02-12T18:22:09.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:22:10.041 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-12T18:22:10.042 INFO:tasks.workunit.client.0.trial116.stdout:start osd.1 2026-02-12T18:22:10.042 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:22:10.043 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:22:10.043 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:22:10.043 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:22:10.056 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:10.052+0000 7f3efbc2c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:10.057 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:10.053+0000 7f3efbc2c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:10.058 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:10.054+0000 7f3efbc2c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:10.187 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:10.183+0000 7f3efbc2c780 -1 Falling back to public interface 2026-02-12T18:22:10.270 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-12T18:22:10.270 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:22:10.270 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T18:22:10.270 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:22:10.270 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:22:10.270 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:22:10.271 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:22:10.271 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:22:10.271 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:22:10.271 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:22:10.477 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:10.473+0000 7f3efbc2c780 -1 osd.1 0 log_to_monitors true 2026-02-12T18:22:10.498 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:22:11.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:22:11.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:22:11.499 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:22:11.500 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:22:11.500 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:22:11.500 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:22:11.725 INFO:tasks.workunit.client.0.trial116.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1261857779,v1:127.0.0.1:6811/1261857779] [v2:127.0.0.1:6812/1261857779,v1:127.0.0.1:6813/1261857779] exists,up b2d08ec4-55c3-47fb-a37a-56257fef04aa 2026-02-12T18:22:11.728 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:22:11.728 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:22:11.728 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:22:11.728 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:22:11.728 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-12T18:22:11.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:22:11.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:22:11.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-12T18:22:11.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:22:11.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-12T18:22:11.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:22:11.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:22:11.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:22:11.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:22:11.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-12T18:22:11.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-12T18:22:11.730 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:22:11.730 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:22:11.730 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:22:11.730 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:22:11.730 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:22:11.730 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:22:11.730 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:22:11.730 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:22:11.730 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:22:11.730 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:22:11.730 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:22:11.730 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:22:11.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:22:11.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:22:11.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:22:11.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:22:11.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:22:11.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:22:11.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:22:11.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:22:11.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:22:11.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:22:11.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-12T18:22:11.731 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:22:11.732 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=75dd7127-4d8b-4ec5-84f7-bafdab5d91a5 2026-02-12T18:22:11.732 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 75dd7127-4d8b-4ec5-84f7-bafdab5d91a5' 2026-02-12T18:22:11.733 INFO:tasks.workunit.client.0.trial116.stdout:add osd2 75dd7127-4d8b-4ec5-84f7-bafdab5d91a5 2026-02-12T18:22:11.733 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:22:11.739 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBTGo5pKincKxAAL58zNzC3SuVcJCTTaePlbQ== 2026-02-12T18:22:11.739 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBTGo5pKincKxAAL58zNzC3SuVcJCTTaePlbQ=="}' 2026-02-12T18:22:11.739 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 75dd7127-4d8b-4ec5-84f7-bafdab5d91a5 -i td/test-erasure-eio/2/new.json 2026-02-12T18:22:11.969 INFO:tasks.workunit.client.0.trial116.stdout:2 2026-02-12T18:22:11.977 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-12T18:22:11.978 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBTGo5pKincKxAAL58zNzC3SuVcJCTTaePlbQ== --osd-uuid 75dd7127-4d8b-4ec5-84f7-bafdab5d91a5 2026-02-12T18:22:11.993 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:11.990+0000 7fbf7c80f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:11.995 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:11.991+0000 7fbf7c80f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:11.995 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:11.992+0000 7fbf7c80f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:11.996 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:11.992+0000 7fbf7c80f780 -1 bdev(0x3c8ac00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-12T18:22:11.996 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:11.992+0000 7fbf7c80f780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-12T18:22:12.309 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-12T18:22:12.309 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:22:12.310 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-12T18:22:12.310 INFO:tasks.workunit.client.0.trial116.stdout:adding osd2 key to auth repository 2026-02-12T18:22:12.310 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:22:12.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-12T18:22:12.630 INFO:tasks.workunit.client.0.trial116.stdout:start osd.2 2026-02-12T18:22:12.631 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:22:12.631 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:22:12.631 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:22:12.631 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:22:12.644 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:12.640+0000 7f1cbbc5d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:12.646 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:12.642+0000 7f1cbbc5d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:12.647 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:12.643+0000 7f1cbbc5d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:12.763 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:12.760+0000 7f1cbbc5d780 -1 Falling back to public interface 2026-02-12T18:22:12.856 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-12T18:22:12.856 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:22:12.856 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T18:22:12.856 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:22:12.856 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:22:12.856 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:22:12.856 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:22:12.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:22:12.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:22:12.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:22:13.082 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:22:13.083 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:13.079+0000 7f1cbbc5d780 -1 osd.2 0 log_to_monitors true 2026-02-12T18:22:14.084 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:22:14.084 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:22:14.084 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:22:14.085 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:22:14.085 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:22:14.085 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:22:14.316 INFO:tasks.workunit.client.0.trial116.stdout:osd.2 up in weight 1 up_from 18 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3542283807,v1:127.0.0.1:6819/3542283807] [v2:127.0.0.1:6820/3542283807,v1:127.0.0.1:6821/3542283807] exists,up 75dd7127-4d8b-4ec5-84f7-bafdab5d91a5 2026-02-12T18:22:14.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:22:14.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:22:14.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:22:14.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:22:14.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-12T18:22:14.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:22:14.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:22:14.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-12T18:22:14.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:22:14.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-12T18:22:14.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:22:14.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:22:14.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:22:14.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:22:14.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-12T18:22:14.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-12T18:22:14.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:22:14.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:22:14.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:22:14.321 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:22:14.321 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:22:14.321 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:22:14.322 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:22:14.322 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:22:14.322 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:22:14.322 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:22:14.322 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:22:14.322 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:22:14.322 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:22:14.322 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:22:14.322 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:22:14.322 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:22:14.322 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:22:14.322 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:22:14.323 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:22:14.323 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:22:14.323 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:22:14.323 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:22:14.323 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-12T18:22:14.323 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:22:14.323 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0ae9377b-3b94-42d9-86e5-8bf25dbb6af4 2026-02-12T18:22:14.323 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 0ae9377b-3b94-42d9-86e5-8bf25dbb6af4' 2026-02-12T18:22:14.323 INFO:tasks.workunit.client.0.trial116.stdout:add osd3 0ae9377b-3b94-42d9-86e5-8bf25dbb6af4 2026-02-12T18:22:14.324 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:22:14.329 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBWGo5p5310ExAAdy7gqKJBZDIq9pqH4NhLWQ== 2026-02-12T18:22:14.329 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBWGo5p5310ExAAdy7gqKJBZDIq9pqH4NhLWQ=="}' 2026-02-12T18:22:14.329 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0ae9377b-3b94-42d9-86e5-8bf25dbb6af4 -i td/test-erasure-eio/3/new.json 2026-02-12T18:22:14.559 INFO:tasks.workunit.client.0.trial116.stdout:3 2026-02-12T18:22:14.567 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-12T18:22:14.568 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBWGo5p5310ExAAdy7gqKJBZDIq9pqH4NhLWQ== --osd-uuid 0ae9377b-3b94-42d9-86e5-8bf25dbb6af4 2026-02-12T18:22:14.584 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:14.580+0000 7fd1a0258780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:14.585 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:14.582+0000 7fd1a0258780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:14.586 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:14.583+0000 7fd1a0258780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:14.587 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:14.583+0000 7fd1a0258780 -1 bdev(0x34f2c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-12T18:22:14.587 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:14.583+0000 7fd1a0258780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-12T18:22:14.900 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-12T18:22:14.900 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:22:14.900 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-12T18:22:14.901 INFO:tasks.workunit.client.0.trial116.stdout:adding osd3 key to auth repository 2026-02-12T18:22:14.901 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:22:15.218 INFO:tasks.workunit.client.0.trial116.stdout:start osd.3 2026-02-12T18:22:15.219 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-12T18:22:15.219 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:22:15.219 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:22:15.219 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:22:15.219 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:22:15.233 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:15.229+0000 7fa7e6e15780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:15.234 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:15.231+0000 7fa7e6e15780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:15.235 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:15.231+0000 7fa7e6e15780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:15.358 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:15.355+0000 7fa7e6e15780 -1 Falling back to public interface 2026-02-12T18:22:15.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-12T18:22:15.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:22:15.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-12T18:22:15.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:22:15.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:22:15.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:22:15.443 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:22:15.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:22:15.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:22:15.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:22:15.662 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:15.658+0000 7fa7e6e15780 -1 osd.3 0 log_to_monitors true 2026-02-12T18:22:15.671 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:22:16.673 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:22:16.673 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:22:16.673 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:22:16.673 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:22:16.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:22:16.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:22:16.909 INFO:tasks.workunit.client.0.trial116.stdout:osd.3 up in weight 1 up_from 26 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3842348233,v1:127.0.0.1:6827/3842348233] [v2:127.0.0.1:6828/3842348233,v1:127.0.0.1:6829/3842348233] exists,up 0ae9377b-3b94-42d9-86e5-8bf25dbb6af4 2026-02-12T18:22:16.911 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:22:16.911 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:22:16.911 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:22:16.911 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-12T18:22:16.911 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T18:22:16.911 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T18:22:16.911 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:22:16.911 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:22:16.911 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:22:16.912 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.0.asok 2026-02-12T18:22:16.912 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-12T18:22:16.912 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-osd.0.asok log flush 2026-02-12T18:22:16.955 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:22:16.958 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-12T18:22:16.961 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:22:07.927+0000 7f58a8e29780 0 load: jerasure load: lrc 2026-02-12T18:22:16.962 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:325: TEST_rados_get_bad_size_shard_0: local poolname=pool-jerasure 2026-02-12T18:22:16.962 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:326: TEST_rados_get_bad_size_shard_0: create_erasure_coded_pool pool-jerasure 2 1 2026-02-12T18:22:16.962 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-12T18:22:16.962 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-12T18:22:16.962 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-12T18:22:16.962 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-12T18:22:16.963 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-12T18:22:16.963 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-12T18:22:16.963 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-12T18:22:17.253 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-12T18:22:17.253 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-12T18:22:17.625 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-jerasure' already exists 2026-02-12T18:22:17.633 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:22:18.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-12T18:22:18.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:22:18.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:22:18.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:22:18.636 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:22:18.636 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:22:18.637 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:22:18.637 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:22:18.637 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:22:18.637 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:22:18.724 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:22:18.724 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:22:18.724 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:22:18.725 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:22:18.725 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:22:18.725 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:22:18.952 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:22:18.952 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:22:18.952 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:22:18.952 INFO:tasks.workunit.client.0.trial116.stderr:3' 2026-02-12T18:22:18.952 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:22:18.953 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:18.953 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:22:19.018 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803781 2026-02-12T18:22:19.018 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803781 2026-02-12T18:22:19.019 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781' 2026-02-12T18:22:19.019 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:19.019 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:22:19.082 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574853 2026-02-12T18:22:19.082 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574853 2026-02-12T18:22:19.082 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-55834574853' 2026-02-12T18:22:19.082 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:19.082 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:22:19.147 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411332 2026-02-12T18:22:19.147 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411332 2026-02-12T18:22:19.147 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-55834574853 2-77309411332' 2026-02-12T18:22:19.147 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:19.147 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:22:19.210 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149699 2026-02-12T18:22:19.211 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149699 2026-02-12T18:22:19.211 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-55834574853 2-77309411332 3-111669149699' 2026-02-12T18:22:19.211 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:22:19.211 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803781 2026-02-12T18:22:19.211 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:22:19.212 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:22:19.212 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803781 2026-02-12T18:22:19.213 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:22:19.213 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803781 2026-02-12T18:22:19.214 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-12T18:22:19.214 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.0 seq 25769803781 2026-02-12T18:22:19.214 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:22:19.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803779 -lt 25769803781 2026-02-12T18:22:19.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:22:20.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:22:20.444 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:22:20.668 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803781 2026-02-12T18:22:20.669 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:22:20.669 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574853 2026-02-12T18:22:20.669 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:22:20.670 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:22:20.671 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574853 2026-02-12T18:22:20.671 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:22:20.672 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574853 2026-02-12T18:22:20.672 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.1 seq 55834574853 2026-02-12T18:22:20.673 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574853' 2026-02-12T18:22:20.673 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:22:20.900 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574853 -lt 55834574853 2026-02-12T18:22:20.900 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:22:20.901 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411332 2026-02-12T18:22:20.901 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:22:20.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:22:20.902 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411332 2026-02-12T18:22:20.902 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:22:20.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411332 2026-02-12T18:22:20.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411332' 2026-02-12T18:22:20.904 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.2 seq 77309411332 2026-02-12T18:22:20.904 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:22:21.127 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411333 -lt 77309411332 2026-02-12T18:22:21.127 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:22:21.128 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149699 2026-02-12T18:22:21.128 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:22:21.128 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:22:21.129 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149699 2026-02-12T18:22:21.129 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:22:21.130 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149699 2026-02-12T18:22:21.130 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149699' 2026-02-12T18:22:21.131 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.3 seq 111669149699 2026-02-12T18:22:21.131 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:22:21.357 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149699 -lt 111669149699 2026-02-12T18:22:21.358 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:22:21.358 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:22:21.358 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:22:21.681 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:22:21.681 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:22:21.683 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:22:21.684 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:22:21.684 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:22:21.684 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:22:21.684 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:22:21.684 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:22:21.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:22:21.905 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:22:21.905 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:22:21.905 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:22:22.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:22:22.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:22:22.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:22:22.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:328: TEST_rados_get_bad_size_shard_0: local shard_id=0 2026-02-12T18:22:22.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:329: TEST_rados_get_bad_size_shard_0: rados_get_data_bad_size td/test-erasure-eio 0 10 2026-02-12T18:22:22.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-12T18:22:22.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-12T18:22:22.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=0 2026-02-12T18:22:22.224 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-12T18:22:22.224 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=10 2026-02-12T18:22:22.224 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-12T18:22:22.224 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-12T18:22:22.224 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-12T18:22:22.224 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-76821-0-10 2026-02-12T18:22:22.224 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-76821-0-10 2026-02-12T18:22:22.224 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-12T18:22:22.224 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-12T18:22:22.224 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-76821-0-10 2026-02-12T18:22:22.224 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:22:22.224 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-12T18:22:22.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:22:22.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-12T18:22:22.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:22:22.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-12T18:22:22.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:22:22.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-12T18:22:22.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-76821-0-10 td/test-erasure-eio/ORIGINAL 2026-02-12T18:22:22.240 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-76821-0-10 td/test-erasure-eio 0 10 set 2026-02-12T18:22:22.240 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-76821-0-10 2026-02-12T18:22:22.240 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-12T18:22:22.240 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-12T18:22:22.240 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-12T18:22:22.240 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=0 2026-02-12T18:22:22.240 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-12T18:22:22.240 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-12T18:22:22.241 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-12T18:22:22.241 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-12T18:22:22.241 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-12T18:22:22.241 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-76821-0-10 2026-02-12T18:22:22.241 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:22:22.241 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-76821-0-10 2026-02-12T18:22:22.241 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-76821-0-10 2026-02-12T18:22:22.241 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:22:22.470 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:22:22.470 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:22:22.470 INFO:tasks.workunit.client.0.trial116.stderr:2' 2026-02-12T18:22:22.470 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-12T18:22:22.470 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-12T18:22:22.470 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-12T18:22:22.470 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=3 2026-02-12T18:22:22.470 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-12T18:22:22.751 INFO:tasks.workunit.client.0.trial116.stderr:noout is set 2026-02-12T18:22:22.759 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-12T18:22:22.760 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-12T18:22:22.760 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-12T18:22:22.762 INFO:tasks.workunit.client.0.trial116.stderr:1+0 records in 2026-02-12T18:22:22.762 INFO:tasks.workunit.client.0.trial116.stderr:1+0 records out 2026-02-12T18:22:22.762 INFO:tasks.workunit.client.0.trial116.stderr:10 bytes copied, 4.6957e-05 s, 213 kB/s 2026-02-12T18:22:22.762 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-76821-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:22:22.762 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-12T18:22:22.762 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-12T18:22:22.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-12T18:22:22.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-12T18:22:22.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-76821-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:22:22.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-12T18:22:22.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-12T18:22:22.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-12T18:22:22.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-12T18:22:22.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-12T18:22:22.763 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:22:22.763 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:22:22.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:22:22.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:22:22.764 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:22:23.069 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:22:23.069 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-76821-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:22:23.069 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-12T18:22:23.070 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-12T18:22:23.070 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-12T18:22:23.070 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-12T18:22:23.070 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-12T18:22:23.070 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-76821-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:22:23.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-12T18:22:23.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-12T18:22:23.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T18:22:23.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-12T18:22:23.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T18:22:23.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-12T18:22:23.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:22:23.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:22:23.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:22:23.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:22:23.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-12T18:22:23.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-12T18:22:23.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T18:22:23.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T18:22:23.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:22:23.224 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T18:22:23.224 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:22:23.224 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:22:23.224 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:22:23.224 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:22:23.224 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:22:23.224 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:22:23.224 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:22:23.224 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:22:23.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:22:23.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:22:23.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:22:23.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:22:23.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:22:23.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:22:23.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T18:22:23.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-12T18:22:23.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-12T18:22:23.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-12T18:22:23.225 INFO:tasks.workunit.client.0.trial116.stderr:start osd.3 2026-02-12T18:22:23.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:22:23.226 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-12T18:22:23.226 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-12T18:22:23.227 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T18:22:23.227 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T18:22:23.227 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T18:22:23.240 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:23.237+0000 7f51b2bdd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:23.241 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:23.238+0000 7f51b2bdd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:23.242 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:23.239+0000 7f51b2bdd780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:23.377 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:23.374+0000 7f51b2bdd780 -1 Falling back to public interface 2026-02-12T18:22:23.452 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-12T18:22:23.452 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:22:23.452 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-12T18:22:23.452 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:22:23.452 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:22:23.453 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:22:23.453 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:22:23.453 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:22:23.453 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:22:23.453 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:22:23.660 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:23.657+0000 7f51b2bdd780 -1 osd.3 35 log_to_monitors true 2026-02-12T18:22:23.679 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:22:24.680 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:22:24.681 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:22:24.681 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:22:24.681 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:22:24.681 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:22:24.681 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:22:24.906 INFO:tasks.workunit.client.0.trial116.stderr:osd.3 up in weight 1 up_from 40 up_thru 40 down_at 37 last_clean_interval [26,35) [v2:127.0.0.1:6826/4131550418,v1:127.0.0.1:6827/4131550418] [v2:127.0.0.1:6828/4131550418,v1:127.0.0.1:6829/4131550418] exists,up 0ae9377b-3b94-42d9-86e5-8bf25dbb6af4 2026-02-12T18:22:24.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:22:24.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:22:24.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:22:24.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-12T18:22:24.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:22:24.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:22:24.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:22:24.906 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:22:24.907 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:22:24.907 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:22:24.907 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:22:24.908 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:22:24.908 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:22:24.992 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:22:24.992 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:22:24.992 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:22:24.992 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:22:24.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:22:24.993 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:22:25.219 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:22:25.219 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:22:25.220 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:22:25.220 INFO:tasks.workunit.client.0.trial116.stderr:3' 2026-02-12T18:22:25.220 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:22:25.220 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:25.220 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:22:25.284 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803785 2026-02-12T18:22:25.284 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803785 2026-02-12T18:22:25.284 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785' 2026-02-12T18:22:25.284 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:25.284 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:22:25.348 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574856 2026-02-12T18:22:25.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574856 2026-02-12T18:22:25.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574856' 2026-02-12T18:22:25.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:25.349 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:22:25.414 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411336 2026-02-12T18:22:25.414 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411336 2026-02-12T18:22:25.414 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574856 2-77309411336' 2026-02-12T18:22:25.414 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:25.414 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:22:25.478 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=171798691843 2026-02-12T18:22:25.478 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 171798691843 2026-02-12T18:22:25.478 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803785 1-55834574856 2-77309411336 3-171798691843' 2026-02-12T18:22:25.478 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:22:25.479 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803785 2026-02-12T18:22:25.479 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:22:25.479 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:22:25.480 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803785 2026-02-12T18:22:25.480 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:22:25.481 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803785 2026-02-12T18:22:25.482 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803785' 2026-02-12T18:22:25.482 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.0 seq 25769803785 2026-02-12T18:22:25.482 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:22:25.707 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803785 2026-02-12T18:22:25.707 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:22:26.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:22:26.709 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:22:26.936 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803785 -lt 25769803785 2026-02-12T18:22:26.936 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:22:26.937 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574856 2026-02-12T18:22:26.937 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:22:26.938 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:22:26.938 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574856 2026-02-12T18:22:26.938 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:22:26.939 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574856 2026-02-12T18:22:26.939 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574856' 2026-02-12T18:22:26.940 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.1 seq 55834574856 2026-02-12T18:22:26.940 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:22:27.175 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574856 -lt 55834574856 2026-02-12T18:22:27.176 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:22:27.176 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411336 2026-02-12T18:22:27.176 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:22:27.177 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:22:27.178 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411336 2026-02-12T18:22:27.178 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:22:27.179 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411336 2026-02-12T18:22:27.179 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411336' 2026-02-12T18:22:27.179 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.2 seq 77309411336 2026-02-12T18:22:27.179 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:22:27.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411336 -lt 77309411336 2026-02-12T18:22:27.410 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:22:27.410 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-171798691843 2026-02-12T18:22:27.410 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:22:27.411 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:22:27.412 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-171798691843 2026-02-12T18:22:27.412 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:22:27.413 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=171798691843 2026-02-12T18:22:27.413 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 171798691843' 2026-02-12T18:22:27.413 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.3 seq 171798691843 2026-02-12T18:22:27.413 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:22:27.643 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 171798691843 -lt 171798691843 2026-02-12T18:22:27.644 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:22:27.644 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:22:27.644 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:22:27.977 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:22:27.977 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:22:27.977 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:22:27.977 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:22:27.977 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:22:27.977 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:22:27.977 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:22:27.977 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:22:28.199 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:22:28.200 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:22:28.200 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:22:28.200 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:22:28.518 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:22:28.518 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:22:28.518 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:22:28.518 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-12T18:22:28.518 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-12T18:22:28.833 INFO:tasks.workunit.client.0.trial116.stderr:noout is unset 2026-02-12T18:22:28.845 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-76821-0-10 2026-02-12T18:22:28.845 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-12T18:22:28.845 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-12T18:22:28.845 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-76821-0-10 2026-02-12T18:22:28.846 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-12T18:22:28.846 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-12T18:22:28.846 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-76821-0-10 td/test-erasure-eio/COPY 2026-02-12T18:22:28.864 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-12T18:22:28.865 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-12T18:22:28.866 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 0 + 1 2026-02-12T18:22:28.868 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=1 2026-02-12T18:22:28.868 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-76821-0-10 td/test-erasure-eio 1 10 set 2026-02-12T18:22:28.868 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-76821-0-10 2026-02-12T18:22:28.868 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-12T18:22:28.868 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-12T18:22:28.868 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-12T18:22:28.868 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-12T18:22:28.868 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-12T18:22:28.869 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-12T18:22:28.869 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-12T18:22:28.869 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-12T18:22:28.869 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-12T18:22:28.869 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-76821-0-10 2026-02-12T18:22:28.869 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:22:28.869 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-76821-0-10 2026-02-12T18:22:28.870 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-76821-0-10 2026-02-12T18:22:28.870 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:22:29.093 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:22:29.093 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:22:29.093 INFO:tasks.workunit.client.0.trial116.stderr:2' 2026-02-12T18:22:29.093 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-12T18:22:29.093 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-12T18:22:29.093 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-12T18:22:29.094 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-12T18:22:29.094 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-12T18:22:29.412 INFO:tasks.workunit.client.0.trial116.stderr:noout is set 2026-02-12T18:22:29.421 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-12T18:22:29.421 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-12T18:22:29.421 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-12T18:22:29.424 INFO:tasks.workunit.client.0.trial116.stderr:1+0 records in 2026-02-12T18:22:29.424 INFO:tasks.workunit.client.0.trial116.stderr:1+0 records out 2026-02-12T18:22:29.424 INFO:tasks.workunit.client.0.trial116.stderr:10 bytes copied, 3.6327e-05 s, 275 kB/s 2026-02-12T18:22:29.424 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-76821-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:22:29.424 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-12T18:22:29.424 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-12T18:22:29.424 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-12T18:22:29.424 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-12T18:22:29.424 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-76821-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:22:29.424 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-12T18:22:29.425 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-12T18:22:29.425 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-12T18:22:29.425 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-12T18:22:29.425 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-12T18:22:29.425 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:22:29.425 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:22:29.425 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:22:29.425 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:22:29.425 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:22:29.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:22:29.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-76821-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:22:29.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-12T18:22:29.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-12T18:22:29.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-12T18:22:29.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-12T18:22:29.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-12T18:22:29.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-76821-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:22:29.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-12T18:22:29.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-12T18:22:29.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T18:22:29.882 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-12T18:22:29.882 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T18:22:29.882 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-12T18:22:29.882 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:22:29.882 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:22:29.882 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:22:29.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:22:29.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-12T18:22:29.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-12T18:22:29.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T18:22:29.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T18:22:29.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:22:29.883 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T18:22:29.884 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:22:29.884 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:22:29.884 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:22:29.884 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:22:29.884 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:22:29.884 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:22:29.884 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:22:29.885 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:22:29.885 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:22:29.885 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:22:29.885 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:22:29.885 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:22:29.885 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:22:29.885 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:22:29.886 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T18:22:29.886 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-12T18:22:29.886 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-12T18:22:29.886 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-12T18:22:29.886 INFO:tasks.workunit.client.0.trial116.stderr:start osd.1 2026-02-12T18:22:29.886 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:22:29.886 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-12T18:22:29.887 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-12T18:22:29.887 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T18:22:29.887 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T18:22:29.887 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T18:22:29.893 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:29.889+0000 7f8272a34780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:29.894 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:29.891+0000 7f8272a34780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:29.895 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:29.892+0000 7f8272a34780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:30.032 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:30.029+0000 7f8272a34780 -1 Falling back to public interface 2026-02-12T18:22:30.107 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-12T18:22:30.107 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:22:30.107 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T18:22:30.107 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:22:30.107 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:22:30.108 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:22:30.108 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:22:30.108 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:22:30.108 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:22:30.108 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:22:30.334 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:22:30.384 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:30.381+0000 7f8272a34780 -1 osd.1 43 log_to_monitors true 2026-02-12T18:22:31.336 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:22:31.336 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:22:31.337 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:22:31.337 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:22:31.337 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:22:31.337 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:22:31.564 INFO:tasks.workunit.client.0.trial116.stderr:osd.1 up in weight 1 up_from 49 up_thru 49 down_at 46 last_clean_interval [13,43) [v2:127.0.0.1:6810/2122268100,v1:127.0.0.1:6811/2122268100] [v2:127.0.0.1:6812/2122268100,v1:127.0.0.1:6813/2122268100] exists,up b2d08ec4-55c3-47fb-a37a-56257fef04aa 2026-02-12T18:22:31.565 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:22:31.565 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:22:31.565 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:22:31.565 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-12T18:22:31.565 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:22:31.565 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:22:31.565 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:22:31.565 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:22:31.566 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:22:31.566 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:22:31.566 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:22:31.566 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:22:31.567 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:22:31.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:22:31.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:22:31.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:22:31.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:22:31.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:22:31.651 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:22:31.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:22:31.881 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:22:31.881 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:22:31.881 INFO:tasks.workunit.client.0.trial116.stderr:3' 2026-02-12T18:22:31.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:22:31.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:31.881 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:22:31.946 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803788 2026-02-12T18:22:31.946 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803788 2026-02-12T18:22:31.946 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788' 2026-02-12T18:22:31.946 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:31.947 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:22:32.012 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397507 2026-02-12T18:22:32.012 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397507 2026-02-12T18:22:32.012 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788 1-210453397507' 2026-02-12T18:22:32.012 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:32.012 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:22:32.077 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411339 2026-02-12T18:22:32.077 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411339 2026-02-12T18:22:32.077 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788 1-210453397507 2-77309411339' 2026-02-12T18:22:32.077 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:32.077 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:22:32.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=171798691846 2026-02-12T18:22:32.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 171798691846 2026-02-12T18:22:32.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803788 1-210453397507 2-77309411339 3-171798691846' 2026-02-12T18:22:32.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:22:32.143 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803788 2026-02-12T18:22:32.143 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:22:32.143 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:22:32.143 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803788 2026-02-12T18:22:32.143 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:22:32.145 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803788 2026-02-12T18:22:32.145 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803788' 2026-02-12T18:22:32.145 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.0 seq 25769803788 2026-02-12T18:22:32.145 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:22:32.370 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803786 -lt 25769803788 2026-02-12T18:22:32.371 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:22:33.371 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:22:33.372 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:22:33.597 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803786 -lt 25769803788 2026-02-12T18:22:33.597 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:22:34.598 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-12T18:22:34.599 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:22:34.833 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803788 -lt 25769803788 2026-02-12T18:22:34.833 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:22:34.834 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-210453397507 2026-02-12T18:22:34.834 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:22:34.835 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:22:34.835 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-210453397507 2026-02-12T18:22:34.835 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:22:34.837 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=210453397507 2026-02-12T18:22:34.837 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 210453397507' 2026-02-12T18:22:34.837 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.1 seq 210453397507 2026-02-12T18:22:34.838 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:22:35.067 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 210453397507 -lt 210453397507 2026-02-12T18:22:35.068 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:22:35.068 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411339 2026-02-12T18:22:35.068 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:22:35.069 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:22:35.070 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411339 2026-02-12T18:22:35.070 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:22:35.071 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411339 2026-02-12T18:22:35.071 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411339' 2026-02-12T18:22:35.071 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.2 seq 77309411339 2026-02-12T18:22:35.071 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:22:35.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411339 -lt 77309411339 2026-02-12T18:22:35.298 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:22:35.298 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-171798691846 2026-02-12T18:22:35.298 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:22:35.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:22:35.300 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-171798691846 2026-02-12T18:22:35.300 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:22:35.301 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=171798691846 2026-02-12T18:22:35.301 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 171798691846' 2026-02-12T18:22:35.301 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.3 seq 171798691846 2026-02-12T18:22:35.301 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:22:35.525 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 171798691847 -lt 171798691846 2026-02-12T18:22:35.526 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:22:35.526 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:22:35.526 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:22:35.837 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:22:35.837 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:22:35.837 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:22:35.838 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:22:35.838 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:22:35.838 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:22:35.838 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:22:35.838 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:22:36.058 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:22:36.059 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:22:36.059 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:22:36.059 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:22:36.375 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:22:36.375 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:22:36.375 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:22:36.375 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-12T18:22:36.376 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-12T18:22:36.690 INFO:tasks.workunit.client.0.trial116.stderr:noout is unset 2026-02-12T18:22:36.701 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-12T18:22:36.702 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:262: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-76821-0-10 fail 2026-02-12T18:22:36.702 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-12T18:22:36.702 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-12T18:22:36.702 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-76821-0-10 2026-02-12T18:22:36.702 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-12T18:22:36.702 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-12T18:22:36.702 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-size-76821-0-10 td/test-erasure-eio/COPY 2026-02-12T18:22:36.718 INFO:tasks.workunit.client.0.trial116.stderr:error getting pool-jerasure/obj-size-76821-0-10: (5) Input/output error 2026-02-12T18:22:36.720 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-12T18:22:36.720 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-12T18:22:36.722 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:330: TEST_rados_get_bad_size_shard_0: rados_get_data_bad_size td/test-erasure-eio 0 0 2026-02-12T18:22:36.722 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-12T18:22:36.723 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-12T18:22:36.723 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=0 2026-02-12T18:22:36.723 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-12T18:22:36.723 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=0 2026-02-12T18:22:36.723 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-12T18:22:36.723 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-12T18:22:36.723 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-12T18:22:36.723 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-76821-0-0 2026-02-12T18:22:36.723 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-76821-0-0 2026-02-12T18:22:36.723 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-12T18:22:36.723 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-12T18:22:36.723 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-76821-0-0 2026-02-12T18:22:36.724 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:22:36.724 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-12T18:22:36.724 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:22:36.724 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-12T18:22:36.724 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:22:36.724 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-12T18:22:36.724 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:22:36.724 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-12T18:22:36.724 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-76821-0-0 td/test-erasure-eio/ORIGINAL 2026-02-12T18:22:36.739 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-76821-0-0 td/test-erasure-eio 0 0 set 2026-02-12T18:22:36.739 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-76821-0-0 2026-02-12T18:22:36.739 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-12T18:22:36.739 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-12T18:22:36.739 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-12T18:22:36.740 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=0 2026-02-12T18:22:36.740 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-12T18:22:36.740 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-12T18:22:36.740 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-12T18:22:36.740 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-12T18:22:36.740 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-12T18:22:36.740 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-76821-0-0 2026-02-12T18:22:36.740 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:22:36.740 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-76821-0-0 2026-02-12T18:22:36.740 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-76821-0-0 2026-02-12T18:22:36.740 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:22:36.965 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:22:36.965 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:22:36.965 INFO:tasks.workunit.client.0.trial116.stderr:2' 2026-02-12T18:22:36.965 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-12T18:22:36.965 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-12T18:22:36.965 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-12T18:22:36.965 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=3 2026-02-12T18:22:36.966 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-12T18:22:37.282 INFO:tasks.workunit.client.0.trial116.stderr:noout is set 2026-02-12T18:22:37.293 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-12T18:22:37.293 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-12T18:22:37.293 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-12T18:22:37.295 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-76821-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:22:37.295 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-12T18:22:37.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-12T18:22:37.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-12T18:22:37.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-12T18:22:37.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-76821-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:22:37.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-12T18:22:37.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-12T18:22:37.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-12T18:22:37.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-12T18:22:37.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-12T18:22:37.296 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:22:37.296 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:22:37.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:22:37.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:22:37.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:22:37.603 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:22:37.603 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-76821-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:22:37.603 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-12T18:22:37.603 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-12T18:22:37.603 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-12T18:22:37.603 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-12T18:22:37.603 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-12T18:22:37.603 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-76821-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:22:37.746 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-12T18:22:37.746 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-12T18:22:37.746 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T18:22:37.746 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-12T18:22:37.746 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T18:22:37.746 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-12T18:22:37.746 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:22:37.746 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:22:37.747 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:22:37.747 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:22:37.747 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-12T18:22:37.747 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-12T18:22:37.747 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T18:22:37.747 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T18:22:37.747 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:22:37.747 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T18:22:37.747 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:22:37.747 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:22:37.747 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:22:37.747 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:22:37.748 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:22:37.748 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:22:37.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:22:37.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:22:37.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:22:37.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:22:37.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:22:37.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:22:37.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:22:37.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:22:37.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T18:22:37.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-12T18:22:37.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-12T18:22:37.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-12T18:22:37.749 INFO:tasks.workunit.client.0.trial116.stderr:start osd.3 2026-02-12T18:22:37.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:22:37.749 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-12T18:22:37.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-12T18:22:37.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T18:22:37.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T18:22:37.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T18:22:37.758 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:37.754+0000 7f88e7833780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:37.759 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:37.756+0000 7f88e7833780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:37.760 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:37.757+0000 7f88e7833780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:37.915 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:37.912+0000 7f88e7833780 -1 Falling back to public interface 2026-02-12T18:22:37.976 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-12T18:22:37.976 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:22:37.976 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-12T18:22:37.976 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:22:37.976 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:22:37.976 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:22:37.977 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:22:37.977 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:22:37.977 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:22:37.977 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:22:38.202 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:22:38.219 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:38.216+0000 7f88e7833780 -1 osd.3 54 log_to_monitors true 2026-02-12T18:22:39.203 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:22:39.204 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:22:39.204 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:22:39.204 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:22:39.204 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:22:39.204 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:22:39.431 INFO:tasks.workunit.client.0.trial116.stderr:osd.3 up in weight 1 up_from 59 up_thru 59 down_at 56 last_clean_interval [40,54) [v2:127.0.0.1:6826/1793377515,v1:127.0.0.1:6827/1793377515] [v2:127.0.0.1:6828/1793377515,v1:127.0.0.1:6829/1793377515] exists,up 0ae9377b-3b94-42d9-86e5-8bf25dbb6af4 2026-02-12T18:22:39.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:22:39.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:22:39.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:22:39.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-12T18:22:39.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:22:39.432 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:22:39.432 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:22:39.432 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:22:39.432 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:22:39.432 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:22:39.432 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:22:39.432 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:22:39.432 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:22:39.518 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:22:39.518 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:22:39.518 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:22:39.518 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:22:39.518 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:22:39.518 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:22:39.752 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:22:39.753 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:22:39.753 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:22:39.753 INFO:tasks.workunit.client.0.trial116.stderr:3' 2026-02-12T18:22:39.753 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:22:39.753 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:39.753 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:22:39.817 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803792 2026-02-12T18:22:39.817 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803792 2026-02-12T18:22:39.817 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803792' 2026-02-12T18:22:39.817 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:39.817 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:22:39.880 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397510 2026-02-12T18:22:39.880 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397510 2026-02-12T18:22:39.880 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803792 1-210453397510' 2026-02-12T18:22:39.880 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:39.880 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:22:39.946 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411343 2026-02-12T18:22:39.946 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411343 2026-02-12T18:22:39.947 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803792 1-210453397510 2-77309411343' 2026-02-12T18:22:39.947 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:39.947 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:22:40.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=253403070467 2026-02-12T18:22:40.011 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 253403070467 2026-02-12T18:22:40.011 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803792 1-210453397510 2-77309411343 3-253403070467' 2026-02-12T18:22:40.011 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:22:40.011 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803792 2026-02-12T18:22:40.011 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:22:40.012 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:22:40.013 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803792 2026-02-12T18:22:40.013 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:22:40.014 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803792 2026-02-12T18:22:40.014 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803792' 2026-02-12T18:22:40.014 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.0 seq 25769803792 2026-02-12T18:22:40.014 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:22:40.245 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803792 -lt 25769803792 2026-02-12T18:22:40.245 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:22:40.246 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-210453397510 2026-02-12T18:22:40.246 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:22:40.247 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:22:40.247 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-210453397510 2026-02-12T18:22:40.248 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:22:40.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=210453397510 2026-02-12T18:22:40.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 210453397510' 2026-02-12T18:22:40.249 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.1 seq 210453397510 2026-02-12T18:22:40.249 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:22:40.474 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 210453397508 -lt 210453397510 2026-02-12T18:22:40.475 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:22:41.476 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:22:41.476 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:22:41.703 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 210453397508 -lt 210453397510 2026-02-12T18:22:41.703 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:22:42.704 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-12T18:22:42.704 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:22:42.930 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 210453397511 -lt 210453397510 2026-02-12T18:22:42.931 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:22:42.931 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411343 2026-02-12T18:22:42.931 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:22:42.932 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:22:42.933 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411343 2026-02-12T18:22:42.933 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:22:42.934 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411343 2026-02-12T18:22:42.934 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411343' 2026-02-12T18:22:42.934 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.2 seq 77309411343 2026-02-12T18:22:42.935 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:22:43.168 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411343 -lt 77309411343 2026-02-12T18:22:43.168 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:22:43.169 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-253403070467 2026-02-12T18:22:43.169 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:22:43.170 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:22:43.171 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-253403070467 2026-02-12T18:22:43.171 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:22:43.172 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=253403070467 2026-02-12T18:22:43.172 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 253403070467' 2026-02-12T18:22:43.172 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.3 seq 253403070467 2026-02-12T18:22:43.173 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:22:43.407 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 253403070467 -lt 253403070467 2026-02-12T18:22:43.407 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:22:43.407 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:22:43.408 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:22:43.724 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:22:43.724 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:22:43.725 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:22:43.726 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:22:43.726 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:22:43.726 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:22:43.726 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:22:43.726 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:22:43.947 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:22:43.948 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:22:43.948 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:22:43.948 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:22:44.267 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:22:44.267 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:22:44.267 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:22:44.267 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-12T18:22:44.267 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-12T18:22:44.585 INFO:tasks.workunit.client.0.trial116.stderr:noout is unset 2026-02-12T18:22:44.597 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-76821-0-0 2026-02-12T18:22:44.597 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-12T18:22:44.597 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-12T18:22:44.597 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-76821-0-0 2026-02-12T18:22:44.597 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-12T18:22:44.597 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-12T18:22:44.597 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-76821-0-0 td/test-erasure-eio/COPY 2026-02-12T18:22:44.616 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-12T18:22:44.617 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-12T18:22:44.618 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 0 + 1 2026-02-12T18:22:44.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=1 2026-02-12T18:22:44.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-76821-0-0 td/test-erasure-eio 1 0 set 2026-02-12T18:22:44.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-76821-0-0 2026-02-12T18:22:44.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-12T18:22:44.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-12T18:22:44.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-12T18:22:44.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-12T18:22:44.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-12T18:22:44.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-12T18:22:44.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-12T18:22:44.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-12T18:22:44.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-12T18:22:44.621 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-76821-0-0 2026-02-12T18:22:44.621 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:22:44.621 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-76821-0-0 2026-02-12T18:22:44.621 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-76821-0-0 2026-02-12T18:22:44.621 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:22:44.851 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:22:44.851 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:22:44.851 INFO:tasks.workunit.client.0.trial116.stderr:2' 2026-02-12T18:22:44.852 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-12T18:22:44.852 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-12T18:22:44.852 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-12T18:22:44.852 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-12T18:22:44.852 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-12T18:22:45.164 INFO:tasks.workunit.client.0.trial116.stderr:noout is set 2026-02-12T18:22:45.175 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-12T18:22:45.175 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-12T18:22:45.175 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-12T18:22:45.177 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-76821-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:22:45.177 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-12T18:22:45.178 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-12T18:22:45.178 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-12T18:22:45.178 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-12T18:22:45.178 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-76821-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:22:45.178 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-12T18:22:45.178 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-12T18:22:45.178 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-12T18:22:45.178 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-12T18:22:45.178 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-12T18:22:45.178 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:22:45.178 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:22:45.179 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:22:45.179 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:22:45.179 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:22:45.484 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:22:45.484 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-76821-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:22:45.485 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-12T18:22:45.485 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-12T18:22:45.485 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-12T18:22:45.485 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-12T18:22:45.485 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-12T18:22:45.485 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-76821-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:22:45.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-12T18:22:45.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-12T18:22:45.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T18:22:45.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-12T18:22:45.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T18:22:45.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-12T18:22:45.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:22:45.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:22:45.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:22:45.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:22:45.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-12T18:22:45.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-12T18:22:45.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T18:22:45.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T18:22:45.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:22:45.622 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T18:22:45.622 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:22:45.623 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:22:45.623 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:22:45.623 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:22:45.623 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:22:45.623 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:22:45.623 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:22:45.623 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:22:45.623 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:22:45.623 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:22:45.623 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:22:45.623 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:22:45.623 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:22:45.623 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:22:45.624 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T18:22:45.624 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-12T18:22:45.624 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-12T18:22:45.624 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-12T18:22:45.624 INFO:tasks.workunit.client.0.trial116.stderr:start osd.1 2026-02-12T18:22:45.624 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:22:45.624 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-12T18:22:45.625 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-12T18:22:45.625 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T18:22:45.625 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T18:22:45.625 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T18:22:45.635 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:45.632+0000 7f6100c0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:45.637 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:45.633+0000 7f6100c0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:45.638 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:45.634+0000 7f6100c0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:45.805 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:45.802+0000 7f6100c0f780 -1 Falling back to public interface 2026-02-12T18:22:45.852 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-12T18:22:45.852 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:22:45.852 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T18:22:45.852 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:22:45.852 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:22:45.853 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:22:45.853 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:22:45.853 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:22:45.853 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:22:45.853 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:22:46.077 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:22:46.168 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:46.164+0000 7f6100c0f780 -1 osd.1 62 log_to_monitors true 2026-02-12T18:22:47.079 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:22:47.079 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:22:47.079 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:22:47.079 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:22:47.079 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:22:47.080 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:22:47.305 INFO:tasks.workunit.client.0.trial116.stderr:osd.1 up in weight 1 up_from 68 up_thru 68 down_at 65 last_clean_interval [49,62) [v2:127.0.0.1:6810/2062351064,v1:127.0.0.1:6811/2062351064] [v2:127.0.0.1:6812/2062351064,v1:127.0.0.1:6813/2062351064] exists,up b2d08ec4-55c3-47fb-a37a-56257fef04aa 2026-02-12T18:22:47.305 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:22:47.306 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:22:47.306 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:22:47.306 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-12T18:22:47.306 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:22:47.306 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:22:47.306 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:22:47.306 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:22:47.307 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:22:47.307 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:22:47.307 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:22:47.307 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:22:47.307 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:22:47.393 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:22:47.393 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:22:47.393 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:22:47.393 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:22:47.393 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:22:47.393 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:22:47.618 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:22:47.618 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:22:47.618 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:22:47.618 INFO:tasks.workunit.client.0.trial116.stderr:3' 2026-02-12T18:22:47.618 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:22:47.618 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:47.618 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:22:47.682 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803795 2026-02-12T18:22:47.682 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803795 2026-02-12T18:22:47.682 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803795' 2026-02-12T18:22:47.682 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:47.682 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:22:47.746 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776131 2026-02-12T18:22:47.746 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776131 2026-02-12T18:22:47.746 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803795 1-292057776131' 2026-02-12T18:22:47.747 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:47.747 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:22:47.810 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411346 2026-02-12T18:22:47.810 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411346 2026-02-12T18:22:47.810 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803795 1-292057776131 2-77309411346' 2026-02-12T18:22:47.810 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:47.811 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:22:47.877 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=253403070470 2026-02-12T18:22:47.877 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 253403070470 2026-02-12T18:22:47.878 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803795 1-292057776131 2-77309411346 3-253403070470' 2026-02-12T18:22:47.878 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:22:47.878 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803795 2026-02-12T18:22:47.878 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:22:47.879 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:22:47.879 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803795 2026-02-12T18:22:47.879 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:22:47.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803795 2026-02-12T18:22:47.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803795' 2026-02-12T18:22:47.881 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.0 seq 25769803795 2026-02-12T18:22:47.881 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:22:48.109 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803795 -lt 25769803795 2026-02-12T18:22:48.109 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:22:48.110 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-292057776131 2026-02-12T18:22:48.110 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:22:48.111 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:22:48.112 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-292057776131 2026-02-12T18:22:48.112 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:22:48.113 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=292057776131 2026-02-12T18:22:48.113 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 292057776131' 2026-02-12T18:22:48.113 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.1 seq 292057776131 2026-02-12T18:22:48.114 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:22:48.344 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 292057776131 -lt 292057776131 2026-02-12T18:22:48.344 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:22:48.345 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411346 2026-02-12T18:22:48.345 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:22:48.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:22:48.346 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411346 2026-02-12T18:22:48.346 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:22:48.348 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411346 2026-02-12T18:22:48.348 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411346' 2026-02-12T18:22:48.348 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.2 seq 77309411346 2026-02-12T18:22:48.348 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:22:48.571 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411346 -lt 77309411346 2026-02-12T18:22:48.571 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:22:48.571 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-253403070470 2026-02-12T18:22:48.572 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:22:48.572 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:22:48.573 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-253403070470 2026-02-12T18:22:48.573 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:22:48.574 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=253403070470 2026-02-12T18:22:48.574 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 253403070470' 2026-02-12T18:22:48.574 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.3 seq 253403070470 2026-02-12T18:22:48.575 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:22:48.804 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 253403070468 -lt 253403070470 2026-02-12T18:22:48.804 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:22:49.805 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:22:49.805 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:22:50.032 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 253403070471 -lt 253403070470 2026-02-12T18:22:50.033 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:22:50.033 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:22:50.034 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:22:50.344 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:22:50.344 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:22:50.344 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:22:50.344 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:22:50.344 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:22:50.344 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:22:50.344 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:22:50.345 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:22:50.564 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:22:50.565 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:22:50.565 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:22:50.565 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:22:50.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:22:50.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:22:50.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:22:50.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-12T18:22:50.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-12T18:22:51.197 INFO:tasks.workunit.client.0.trial116.stderr:noout is unset 2026-02-12T18:22:51.206 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-12T18:22:51.206 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:262: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-76821-0-0 fail 2026-02-12T18:22:51.206 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-12T18:22:51.206 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-12T18:22:51.206 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-76821-0-0 2026-02-12T18:22:51.206 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-12T18:22:51.206 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-12T18:22:51.206 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-size-76821-0-0 td/test-erasure-eio/COPY 2026-02-12T18:22:51.224 INFO:tasks.workunit.client.0.trial116.stderr:error getting pool-jerasure/obj-size-76821-0-0: (5) Input/output error 2026-02-12T18:22:51.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-12T18:22:51.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-12T18:22:51.227 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:331: TEST_rados_get_bad_size_shard_0: rados_get_data_bad_size td/test-erasure-eio 0 256 add 2026-02-12T18:22:51.228 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-12T18:22:51.228 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-12T18:22:51.228 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=0 2026-02-12T18:22:51.228 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-12T18:22:51.228 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=256 2026-02-12T18:22:51.228 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-12T18:22:51.228 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=add 2026-02-12T18:22:51.228 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-12T18:22:51.228 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-76821-0-256 2026-02-12T18:22:51.228 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-76821-0-256 2026-02-12T18:22:51.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-12T18:22:51.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-12T18:22:51.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-76821-0-256 2026-02-12T18:22:51.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:22:51.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-12T18:22:51.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:22:51.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-12T18:22:51.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:22:51.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-12T18:22:51.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:22:51.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-12T18:22:51.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-76821-0-256 td/test-erasure-eio/ORIGINAL 2026-02-12T18:22:51.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-76821-0-256 td/test-erasure-eio 0 256 add 2026-02-12T18:22:51.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-76821-0-256 2026-02-12T18:22:51.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-12T18:22:51.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-12T18:22:51.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-12T18:22:51.245 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=0 2026-02-12T18:22:51.245 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-12T18:22:51.245 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-12T18:22:51.245 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-12T18:22:51.245 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-12T18:22:51.245 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-12T18:22:51.245 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-76821-0-256 2026-02-12T18:22:51.245 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:22:51.245 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-76821-0-256 2026-02-12T18:22:51.245 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-76821-0-256 2026-02-12T18:22:51.245 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:22:51.475 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:22:51.475 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:22:51.475 INFO:tasks.workunit.client.0.trial116.stderr:2' 2026-02-12T18:22:51.475 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-12T18:22:51.475 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-12T18:22:51.475 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-12T18:22:51.475 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=3 2026-02-12T18:22:51.475 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-12T18:22:51.756 INFO:tasks.workunit.client.0.trial116.stderr:noout is set 2026-02-12T18:22:51.767 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-12T18:22:51.767 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-76821-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:22:51.767 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-12T18:22:51.767 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-12T18:22:51.767 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-12T18:22:51.767 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-12T18:22:51.767 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-76821-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:22:51.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-12T18:22:51.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-12T18:22:51.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-12T18:22:51.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-12T18:22:51.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-12T18:22:51.768 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:22:51.768 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:22:51.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:22:51.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:22:51.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:22:52.074 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:22:52.074 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-76821-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:22:52.074 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-12T18:22:52.074 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-12T18:22:52.074 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-12T18:22:52.074 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-12T18:22:52.074 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-12T18:22:52.074 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-76821-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:22:52.157 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-12T18:22:52.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-12T18:22:52.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T18:22:52.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-12T18:22:52.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T18:22:52.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-12T18:22:52.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:22:52.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:22:52.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:22:52.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:22:52.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-12T18:22:52.159 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-12T18:22:52.159 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T18:22:52.159 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T18:22:52.159 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:22:52.159 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T18:22:52.159 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:22:52.159 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:22:52.159 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:22:52.159 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:22:52.159 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:22:52.159 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:22:52.159 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:22:52.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:22:52.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:22:52.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:22:52.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:22:52.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:22:52.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:22:52.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:22:52.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T18:22:52.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-12T18:22:52.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-12T18:22:52.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-12T18:22:52.160 INFO:tasks.workunit.client.0.trial116.stderr:start osd.3 2026-02-12T18:22:52.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:22:52.161 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-12T18:22:52.161 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-12T18:22:52.161 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T18:22:52.162 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T18:22:52.162 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T18:22:52.172 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:52.168+0000 7f947e60d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:52.173 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:52.170+0000 7f947e60d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:52.174 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:52.170+0000 7f947e60d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:52.339 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:52.336+0000 7f947e60d780 -1 Falling back to public interface 2026-02-12T18:22:52.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-12T18:22:52.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:22:52.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-12T18:22:52.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:22:52.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:22:52.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:22:52.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:22:52.387 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:22:52.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:22:52.388 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:22:52.612 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:22:52.655 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:52.651+0000 7f947e60d780 -1 osd.3 71 log_to_monitors true 2026-02-12T18:22:53.613 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:22:53.613 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:22:53.613 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:22:53.613 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:22:53.613 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:22:53.613 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:22:53.842 INFO:tasks.workunit.client.0.trial116.stderr:osd.3 up in weight 1 up_from 77 up_thru 77 down_at 74 last_clean_interval [59,71) [v2:127.0.0.1:6826/1854592758,v1:127.0.0.1:6827/1854592758] [v2:127.0.0.1:6828/1854592758,v1:127.0.0.1:6829/1854592758] exists,up 0ae9377b-3b94-42d9-86e5-8bf25dbb6af4 2026-02-12T18:22:53.842 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:22:53.842 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:22:53.843 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:22:53.843 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-12T18:22:53.843 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:22:53.843 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:22:53.843 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:22:53.843 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:22:53.843 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:22:53.843 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:22:53.844 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:22:53.844 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:22:53.844 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:22:53.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:22:53.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:22:53.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:22:53.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:22:53.929 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:22:53.929 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:22:54.149 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:22:54.149 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:22:54.149 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:22:54.149 INFO:tasks.workunit.client.0.trial116.stderr:3' 2026-02-12T18:22:54.149 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:22:54.150 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:54.150 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:22:54.212 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803798 2026-02-12T18:22:54.212 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803798 2026-02-12T18:22:54.212 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803798' 2026-02-12T18:22:54.212 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:54.212 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:22:54.276 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776134 2026-02-12T18:22:54.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776134 2026-02-12T18:22:54.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803798 1-292057776134' 2026-02-12T18:22:54.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:54.277 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:22:54.340 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411350 2026-02-12T18:22:54.340 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411350 2026-02-12T18:22:54.340 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803798 1-292057776134 2-77309411350' 2026-02-12T18:22:54.340 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:22:54.340 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:22:54.404 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=330712481795 2026-02-12T18:22:54.404 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 330712481795 2026-02-12T18:22:54.404 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803798 1-292057776134 2-77309411350 3-330712481795' 2026-02-12T18:22:54.404 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:22:54.405 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803798 2026-02-12T18:22:54.405 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:22:54.406 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:22:54.406 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803798 2026-02-12T18:22:54.406 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:22:54.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803798 2026-02-12T18:22:54.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803798' 2026-02-12T18:22:54.408 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.0 seq 25769803798 2026-02-12T18:22:54.408 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:22:54.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803796 -lt 25769803798 2026-02-12T18:22:54.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:22:55.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:22:55.637 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:22:55.863 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803796 -lt 25769803798 2026-02-12T18:22:55.863 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:22:56.864 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-12T18:22:56.865 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:22:57.094 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803799 -lt 25769803798 2026-02-12T18:22:57.094 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:22:57.094 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-292057776134 2026-02-12T18:22:57.094 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:22:57.095 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:22:57.096 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-292057776134 2026-02-12T18:22:57.096 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:22:57.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=292057776134 2026-02-12T18:22:57.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 292057776134' 2026-02-12T18:22:57.097 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.1 seq 292057776134 2026-02-12T18:22:57.098 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:22:57.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 292057776134 -lt 292057776134 2026-02-12T18:22:57.322 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:22:57.322 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411350 2026-02-12T18:22:57.322 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:22:57.323 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:22:57.324 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411350 2026-02-12T18:22:57.324 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:22:57.325 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411350 2026-02-12T18:22:57.325 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411350' 2026-02-12T18:22:57.325 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.2 seq 77309411350 2026-02-12T18:22:57.326 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:22:57.552 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411350 -lt 77309411350 2026-02-12T18:22:57.552 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:22:57.553 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-330712481795 2026-02-12T18:22:57.553 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:22:57.554 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:22:57.554 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-330712481795 2026-02-12T18:22:57.555 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:22:57.555 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=330712481795 2026-02-12T18:22:57.556 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 330712481795' 2026-02-12T18:22:57.556 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.3 seq 330712481795 2026-02-12T18:22:57.556 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:22:57.793 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 330712481795 -lt 330712481795 2026-02-12T18:22:57.794 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:22:57.794 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:22:57.794 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:22:58.111 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:22:58.111 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:22:58.111 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:22:58.111 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:22:58.112 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:22:58.112 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:22:58.112 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:22:58.112 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:22:58.337 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:22:58.337 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:22:58.338 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:22:58.338 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:22:58.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:22:58.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:22:58.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:22:58.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-12T18:22:58.653 INFO:tasks.workunit.client.0.trial116.stderr:1+0 records in 2026-02-12T18:22:58.653 INFO:tasks.workunit.client.0.trial116.stderr:1+0 records out 2026-02-12T18:22:58.653 INFO:tasks.workunit.client.0.trial116.stderr:256 bytes copied, 2.2031e-05 s, 11.6 MB/s 2026-02-12T18:22:58.653 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-76821-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:22:58.654 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-12T18:22:58.654 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-12T18:22:58.654 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-12T18:22:58.654 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-12T18:22:58.654 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-76821-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:22:58.654 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-12T18:22:58.654 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-12T18:22:58.654 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-12T18:22:58.654 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-12T18:22:58.654 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-12T18:22:58.654 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:22:58.655 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:22:58.655 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:22:58.655 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:22:58.655 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:22:58.759 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:22:58.759 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-76821-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:22:58.759 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-12T18:22:58.759 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-12T18:22:58.759 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-12T18:22:58.759 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-12T18:22:58.759 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-12T18:22:58.759 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-76821-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:22:58.914 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-12T18:22:58.915 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-12T18:22:58.915 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T18:22:58.915 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-12T18:22:58.915 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T18:22:58.916 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-12T18:22:58.916 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:22:58.916 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:22:58.916 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:22:58.916 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:22:58.916 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-12T18:22:58.917 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-12T18:22:58.917 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T18:22:58.917 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T18:22:58.917 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:22:58.917 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T18:22:58.917 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:22:58.917 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:22:58.917 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:22:58.917 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:22:58.917 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:22:58.918 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:22:58.918 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:22:58.918 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:22:58.918 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:22:58.918 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:22:58.918 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:22:58.918 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:22:58.918 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:22:58.918 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:22:58.918 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T18:22:58.918 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-12T18:22:58.918 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-12T18:22:58.919 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-12T18:22:58.919 INFO:tasks.workunit.client.0.trial116.stderr:start osd.3 2026-02-12T18:22:58.919 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:22:58.919 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-12T18:22:58.919 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-12T18:22:58.919 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T18:22:58.919 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T18:22:58.919 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T18:22:58.928 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:58.924+0000 7f138c287780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:58.930 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:58.926+0000 7f138c287780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:58.931 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:58.927+0000 7f138c287780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:22:59.071 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:59.067+0000 7f138c287780 -1 Falling back to public interface 2026-02-12T18:22:59.143 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-12T18:22:59.143 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:22:59.143 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-12T18:22:59.143 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:22:59.143 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:22:59.143 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:22:59.143 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:22:59.143 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:22:59.143 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:22:59.144 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:22:59.368 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:22:59.394 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:22:59.390+0000 7f138c287780 -1 osd.3 79 log_to_monitors true 2026-02-12T18:23:00.370 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:23:00.371 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:23:00.371 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:23:00.371 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:23:00.371 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:23:00.371 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:23:00.614 INFO:tasks.workunit.client.0.trial116.stderr:osd.3 up in weight 1 up_from 83 up_thru 83 down_at 80 last_clean_interval [77,79) [v2:127.0.0.1:6826/1466380966,v1:127.0.0.1:6827/1466380966] [v2:127.0.0.1:6828/1466380966,v1:127.0.0.1:6829/1466380966] exists,up 0ae9377b-3b94-42d9-86e5-8bf25dbb6af4 2026-02-12T18:23:00.614 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:23:00.615 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:23:00.615 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:23:00.615 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-12T18:23:00.615 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:23:00.615 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:23:00.615 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:23:00.616 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:23:00.616 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:23:00.616 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:23:00.616 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:23:00.616 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:23:00.616 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:23:00.698 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:23:00.698 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:23:00.698 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:23:00.698 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:23:00.698 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:23:00.698 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:23:00.927 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:23:00.928 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:23:00.928 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:23:00.928 INFO:tasks.workunit.client.0.trial116.stderr:3' 2026-02-12T18:23:00.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:23:00.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:00.929 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:23:00.990 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803802 2026-02-12T18:23:00.990 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803802 2026-02-12T18:23:00.991 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803802' 2026-02-12T18:23:00.991 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:00.991 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:23:01.057 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776137 2026-02-12T18:23:01.057 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776137 2026-02-12T18:23:01.057 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803802 1-292057776137' 2026-02-12T18:23:01.057 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:01.058 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:23:01.123 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411353 2026-02-12T18:23:01.123 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411353 2026-02-12T18:23:01.123 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803802 1-292057776137 2-77309411353' 2026-02-12T18:23:01.123 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:01.123 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:23:01.185 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=356482285571 2026-02-12T18:23:01.185 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 356482285571 2026-02-12T18:23:01.185 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803802 1-292057776137 2-77309411353 3-356482285571' 2026-02-12T18:23:01.185 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:23:01.186 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803802 2026-02-12T18:23:01.186 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:23:01.187 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:23:01.188 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803802 2026-02-12T18:23:01.188 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:23:01.189 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803802 2026-02-12T18:23:01.189 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803802' 2026-02-12T18:23:01.189 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.0 seq 25769803802 2026-02-12T18:23:01.189 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:23:01.416 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803800 -lt 25769803802 2026-02-12T18:23:01.416 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:23:02.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:23:02.418 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:23:02.648 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803802 -lt 25769803802 2026-02-12T18:23:02.648 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:23:02.648 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-292057776137 2026-02-12T18:23:02.648 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:23:02.649 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:23:02.650 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-292057776137 2026-02-12T18:23:02.650 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:23:02.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=292057776137 2026-02-12T18:23:02.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 292057776137' 2026-02-12T18:23:02.652 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.1 seq 292057776137 2026-02-12T18:23:02.652 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:23:02.877 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 292057776138 -lt 292057776137 2026-02-12T18:23:02.878 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:23:02.878 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411353 2026-02-12T18:23:02.878 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:23:02.879 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:23:02.880 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411353 2026-02-12T18:23:02.880 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:23:02.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411353 2026-02-12T18:23:02.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411353' 2026-02-12T18:23:02.881 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.2 seq 77309411353 2026-02-12T18:23:02.881 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:23:03.111 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411353 -lt 77309411353 2026-02-12T18:23:03.111 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:23:03.112 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-356482285571 2026-02-12T18:23:03.112 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:23:03.113 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:23:03.113 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-356482285571 2026-02-12T18:23:03.113 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:23:03.114 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=356482285571 2026-02-12T18:23:03.114 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 356482285571' 2026-02-12T18:23:03.115 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.3 seq 356482285571 2026-02-12T18:23:03.115 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:23:03.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 356482285571 -lt 356482285571 2026-02-12T18:23:03.346 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:23:03.347 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:23:03.347 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:23:03.658 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:23:03.658 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:23:03.659 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:23:03.659 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:23:03.659 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:23:03.659 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:23:03.659 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:23:03.659 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:23:03.884 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:23:03.884 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:23:03.885 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:23:03.885 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:23:04.196 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:23:04.196 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:23:04.196 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:23:04.196 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-12T18:23:04.196 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-12T18:23:04.509 INFO:tasks.workunit.client.0.trial116.stderr:noout is unset 2026-02-12T18:23:04.517 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-76821-0-256 2026-02-12T18:23:04.517 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-12T18:23:04.517 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-12T18:23:04.518 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-76821-0-256 2026-02-12T18:23:04.518 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-12T18:23:04.518 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-12T18:23:04.518 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-76821-0-256 td/test-erasure-eio/COPY 2026-02-12T18:23:04.535 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-12T18:23:04.536 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-12T18:23:04.537 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 0 + 1 2026-02-12T18:23:04.538 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=1 2026-02-12T18:23:04.539 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-76821-0-256 td/test-erasure-eio 1 256 add 2026-02-12T18:23:04.539 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-76821-0-256 2026-02-12T18:23:04.539 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-12T18:23:04.539 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-12T18:23:04.539 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-12T18:23:04.539 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-12T18:23:04.539 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-12T18:23:04.539 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-12T18:23:04.539 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-12T18:23:04.539 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-12T18:23:04.539 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-12T18:23:04.539 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-76821-0-256 2026-02-12T18:23:04.539 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:23:04.539 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-76821-0-256 2026-02-12T18:23:04.539 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-76821-0-256 2026-02-12T18:23:04.539 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:23:04.772 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:23:04.772 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:23:04.772 INFO:tasks.workunit.client.0.trial116.stderr:2' 2026-02-12T18:23:04.772 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-12T18:23:04.772 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-12T18:23:04.772 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-12T18:23:04.772 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-12T18:23:04.772 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-12T18:23:05.089 INFO:tasks.workunit.client.0.trial116.stderr:noout is set 2026-02-12T18:23:05.100 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-12T18:23:05.100 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-76821-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:23:05.101 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-12T18:23:05.101 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-12T18:23:05.101 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-12T18:23:05.101 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-12T18:23:05.101 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-76821-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:23:05.101 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-12T18:23:05.101 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-12T18:23:05.101 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-12T18:23:05.101 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-12T18:23:05.101 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-12T18:23:05.101 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:23:05.101 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:23:05.102 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:23:05.102 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:23:05.102 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:23:05.407 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:23:05.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-76821-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:23:05.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-12T18:23:05.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-12T18:23:05.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-12T18:23:05.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-12T18:23:05.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-12T18:23:05.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-76821-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:23:05.492 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-12T18:23:05.492 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-12T18:23:05.492 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T18:23:05.492 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-12T18:23:05.492 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T18:23:05.492 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-12T18:23:05.492 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:23:05.493 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:23:05.493 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:23:05.493 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:23:05.493 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-12T18:23:05.493 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-12T18:23:05.493 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T18:23:05.493 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T18:23:05.493 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:23:05.493 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T18:23:05.493 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:23:05.493 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:23:05.493 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:23:05.493 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:23:05.494 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:23:05.494 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:23:05.494 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:23:05.494 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:23:05.494 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:23:05.494 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:23:05.494 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:23:05.494 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:23:05.494 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:23:05.494 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:23:05.494 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T18:23:05.494 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-12T18:23:05.495 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-12T18:23:05.495 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-12T18:23:05.495 INFO:tasks.workunit.client.0.trial116.stderr:start osd.1 2026-02-12T18:23:05.495 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:23:05.495 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-12T18:23:05.496 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-12T18:23:05.496 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T18:23:05.496 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T18:23:05.496 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T18:23:05.507 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:05.504+0000 7f54cf035780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:05.508 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:05.505+0000 7f54cf035780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:05.509 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:05.506+0000 7f54cf035780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:05.656 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:05.654+0000 7f54cf035780 -1 Falling back to public interface 2026-02-12T18:23:05.720 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-12T18:23:05.720 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:23:05.720 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T18:23:05.720 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:23:05.720 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:23:05.720 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:23:05.720 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:23:05.721 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:23:05.721 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:23:05.721 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:23:05.949 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:23:05.986 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:05.983+0000 7f54cf035780 -1 osd.1 86 log_to_monitors true 2026-02-12T18:23:06.950 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:23:06.950 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:23:06.950 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:23:06.950 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:23:06.950 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:23:06.950 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:23:07.175 INFO:tasks.workunit.client.0.trial116.stderr:osd.1 up in weight 1 up_from 92 up_thru 92 down_at 89 last_clean_interval [68,86) [v2:127.0.0.1:6810/217054679,v1:127.0.0.1:6811/217054679] [v2:127.0.0.1:6812/217054679,v1:127.0.0.1:6813/217054679] exists,up b2d08ec4-55c3-47fb-a37a-56257fef04aa 2026-02-12T18:23:07.175 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:23:07.175 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:23:07.175 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:23:07.175 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-12T18:23:07.175 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:23:07.175 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:23:07.175 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:23:07.176 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:23:07.176 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:23:07.176 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:23:07.176 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:23:07.176 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:23:07.176 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:23:07.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:23:07.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:23:07.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:23:07.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:23:07.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:23:07.262 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:23:07.489 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:23:07.489 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:23:07.489 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:23:07.489 INFO:tasks.workunit.client.0.trial116.stderr:3' 2026-02-12T18:23:07.489 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:23:07.489 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:07.489 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:23:07.554 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803805 2026-02-12T18:23:07.554 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803805 2026-02-12T18:23:07.554 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803805' 2026-02-12T18:23:07.554 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:07.554 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:23:07.617 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=395136991235 2026-02-12T18:23:07.617 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 395136991235 2026-02-12T18:23:07.617 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803805 1-395136991235' 2026-02-12T18:23:07.617 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:07.617 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:23:07.682 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411356 2026-02-12T18:23:07.682 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411356 2026-02-12T18:23:07.682 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803805 1-395136991235 2-77309411356' 2026-02-12T18:23:07.682 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:07.683 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:23:07.747 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=356482285574 2026-02-12T18:23:07.747 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 356482285574 2026-02-12T18:23:07.747 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803805 1-395136991235 2-77309411356 3-356482285574' 2026-02-12T18:23:07.747 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:23:07.747 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803805 2026-02-12T18:23:07.748 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:23:07.748 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:23:07.749 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803805 2026-02-12T18:23:07.749 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:23:07.750 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803805 2026-02-12T18:23:07.750 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803805' 2026-02-12T18:23:07.750 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.0 seq 25769803805 2026-02-12T18:23:07.751 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:23:07.979 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803805 -lt 25769803805 2026-02-12T18:23:07.979 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:23:07.980 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-395136991235 2026-02-12T18:23:07.980 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:23:07.981 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:23:07.982 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-395136991235 2026-02-12T18:23:07.982 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:23:07.983 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=395136991235 2026-02-12T18:23:07.983 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 395136991235' 2026-02-12T18:23:07.983 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.1 seq 395136991235 2026-02-12T18:23:07.983 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:23:08.213 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 395136991235 -lt 395136991235 2026-02-12T18:23:08.213 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:23:08.214 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411356 2026-02-12T18:23:08.214 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:23:08.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:23:08.216 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411356 2026-02-12T18:23:08.216 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:23:08.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411356 2026-02-12T18:23:08.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411356' 2026-02-12T18:23:08.217 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.2 seq 77309411356 2026-02-12T18:23:08.217 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:23:08.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411356 -lt 77309411356 2026-02-12T18:23:08.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:23:08.444 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-356482285574 2026-02-12T18:23:08.444 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:23:08.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:23:08.445 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-356482285574 2026-02-12T18:23:08.445 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:23:08.446 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=356482285574 2026-02-12T18:23:08.447 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 356482285574' 2026-02-12T18:23:08.447 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.3 seq 356482285574 2026-02-12T18:23:08.447 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:23:08.676 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 356482285574 -lt 356482285574 2026-02-12T18:23:08.677 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:23:08.677 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:23:08.677 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:23:09.011 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:23:09.011 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:23:09.012 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:23:09.012 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:23:09.012 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:23:09.012 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:23:09.012 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:23:09.012 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:23:09.230 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:23:09.230 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:23:09.231 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:23:09.231 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:23:09.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:23:09.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:23:09.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:23:09.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-12T18:23:09.545 INFO:tasks.workunit.client.0.trial116.stderr:1+0 records in 2026-02-12T18:23:09.545 INFO:tasks.workunit.client.0.trial116.stderr:1+0 records out 2026-02-12T18:23:09.546 INFO:tasks.workunit.client.0.trial116.stderr:256 bytes copied, 2.0628e-05 s, 12.4 MB/s 2026-02-12T18:23:09.546 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-76821-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:23:09.546 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-12T18:23:09.546 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-12T18:23:09.546 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-12T18:23:09.546 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-12T18:23:09.546 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-76821-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:23:09.546 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-12T18:23:09.546 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-12T18:23:09.546 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-12T18:23:09.546 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-12T18:23:09.547 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-12T18:23:09.547 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:23:09.547 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:23:09.547 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:23:09.547 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:23:09.547 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:23:09.650 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:23:09.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-76821-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:23:09.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-12T18:23:09.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-12T18:23:09.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-12T18:23:09.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-12T18:23:09.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-12T18:23:09.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-76821-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:23:09.796 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-12T18:23:09.796 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-12T18:23:09.796 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T18:23:09.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-12T18:23:09.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T18:23:09.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-12T18:23:09.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:23:09.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:23:09.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:23:09.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:23:09.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-12T18:23:09.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-12T18:23:09.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T18:23:09.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T18:23:09.798 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:23:09.798 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T18:23:09.798 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:23:09.798 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:23:09.798 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:23:09.798 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:23:09.798 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:23:09.798 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:23:09.798 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:23:09.798 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:23:09.798 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:23:09.798 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:23:09.799 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:23:09.799 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:23:09.799 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:23:09.799 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:23:09.799 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T18:23:09.799 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-12T18:23:09.799 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-12T18:23:09.799 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-12T18:23:09.799 INFO:tasks.workunit.client.0.trial116.stderr:start osd.1 2026-02-12T18:23:09.799 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:23:09.799 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-12T18:23:09.800 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-12T18:23:09.800 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T18:23:09.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T18:23:09.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T18:23:09.810 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:09.807+0000 7f798be27780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:09.811 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:09.808+0000 7f798be27780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:09.812 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:09.809+0000 7f798be27780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:09.951 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:09.948+0000 7f798be27780 -1 Falling back to public interface 2026-02-12T18:23:10.025 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-12T18:23:10.026 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:23:10.026 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T18:23:10.026 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:23:10.026 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:23:10.026 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:23:10.026 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:23:10.026 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:23:10.026 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:23:10.026 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:23:10.259 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:23:10.266 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:10.263+0000 7f798be27780 -1 osd.1 93 log_to_monitors true 2026-02-12T18:23:11.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:23:11.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:23:11.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:23:11.262 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:23:11.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:23:11.262 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:23:11.489 INFO:tasks.workunit.client.0.trial116.stderr:osd.1 up in weight 1 up_from 97 up_thru 97 down_at 94 last_clean_interval [92,93) [v2:127.0.0.1:6810/3552051446,v1:127.0.0.1:6811/3552051446] [v2:127.0.0.1:6812/3552051446,v1:127.0.0.1:6813/3552051446] exists,up b2d08ec4-55c3-47fb-a37a-56257fef04aa 2026-02-12T18:23:11.489 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:23:11.489 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:23:11.490 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:23:11.490 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-12T18:23:11.490 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:23:11.490 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:23:11.490 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:23:11.490 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:23:11.490 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:23:11.490 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:23:11.490 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:23:11.490 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:23:11.490 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:23:11.576 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:23:11.576 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:23:11.576 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:23:11.577 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:23:11.577 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:23:11.577 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:23:11.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:23:11.801 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:23:11.801 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:23:11.801 INFO:tasks.workunit.client.0.trial116.stderr:3' 2026-02-12T18:23:11.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:23:11.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:11.802 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:23:11.865 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803808 2026-02-12T18:23:11.865 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803808 2026-02-12T18:23:11.865 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803808' 2026-02-12T18:23:11.865 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:11.866 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:23:11.931 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=416611827715 2026-02-12T18:23:11.931 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 416611827715 2026-02-12T18:23:11.931 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803808 1-416611827715' 2026-02-12T18:23:11.931 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:11.931 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:23:11.996 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411359 2026-02-12T18:23:11.996 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411359 2026-02-12T18:23:11.996 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803808 1-416611827715 2-77309411359' 2026-02-12T18:23:11.997 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:11.997 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:23:12.060 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=356482285577 2026-02-12T18:23:12.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 356482285577 2026-02-12T18:23:12.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803808 1-416611827715 2-77309411359 3-356482285577' 2026-02-12T18:23:12.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:23:12.061 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803808 2026-02-12T18:23:12.062 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:23:12.062 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:23:12.062 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803808 2026-02-12T18:23:12.063 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:23:12.064 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803808 2026-02-12T18:23:12.064 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803808' 2026-02-12T18:23:12.064 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.0 seq 25769803808 2026-02-12T18:23:12.064 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:23:12.289 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803807 -lt 25769803808 2026-02-12T18:23:12.290 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:23:13.290 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:23:13.291 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:23:13.521 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803807 -lt 25769803808 2026-02-12T18:23:13.521 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:23:14.522 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-12T18:23:14.522 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:23:14.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803808 -lt 25769803808 2026-02-12T18:23:14.749 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:23:14.749 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-416611827715 2026-02-12T18:23:14.750 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:23:14.750 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:23:14.751 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-416611827715 2026-02-12T18:23:14.751 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:23:14.752 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=416611827715 2026-02-12T18:23:14.752 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 416611827715' 2026-02-12T18:23:14.753 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.1 seq 416611827715 2026-02-12T18:23:14.753 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:23:14.982 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 416611827715 -lt 416611827715 2026-02-12T18:23:14.983 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:23:14.984 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411359 2026-02-12T18:23:14.984 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:23:14.984 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:23:14.985 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411359 2026-02-12T18:23:14.985 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:23:14.986 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411359 2026-02-12T18:23:14.987 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411359' 2026-02-12T18:23:14.987 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.2 seq 77309411359 2026-02-12T18:23:14.987 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:23:15.212 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411359 -lt 77309411359 2026-02-12T18:23:15.212 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:23:15.212 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-356482285577 2026-02-12T18:23:15.213 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:23:15.214 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:23:15.214 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-356482285577 2026-02-12T18:23:15.214 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:23:15.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=356482285577 2026-02-12T18:23:15.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 356482285577' 2026-02-12T18:23:15.215 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.3 seq 356482285577 2026-02-12T18:23:15.216 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:23:15.439 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 356482285577 -lt 356482285577 2026-02-12T18:23:15.440 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:23:15.440 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:23:15.440 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:23:15.757 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:23:15.757 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:23:15.757 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:23:15.757 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:23:15.757 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:23:15.757 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:23:15.758 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:23:15.758 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:23:15.980 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:23:15.981 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:23:15.981 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:23:15.981 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:23:16.301 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:23:16.301 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:23:16.301 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:23:16.301 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-12T18:23:16.302 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-12T18:23:16.618 INFO:tasks.workunit.client.0.trial116.stderr:noout is unset 2026-02-12T18:23:16.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-12T18:23:16.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:262: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-76821-0-256 fail 2026-02-12T18:23:16.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-12T18:23:16.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-12T18:23:16.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-76821-0-256 2026-02-12T18:23:16.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-12T18:23:16.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-12T18:23:16.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-size-76821-0-256 td/test-erasure-eio/COPY 2026-02-12T18:23:16.643 INFO:tasks.workunit.client.0.trial116.stderr:error getting pool-jerasure/obj-size-76821-0-256: (5) Input/output error 2026-02-12T18:23:16.645 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-12T18:23:16.645 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-12T18:23:16.646 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:332: TEST_rados_get_bad_size_shard_0: delete_erasure_coded_pool pool-jerasure 2026-02-12T18:23:16.646 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-12T18:23:16.646 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-12T18:23:16.884 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-jerasure' does not exist 2026-02-12T18:23:16.893 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-12T18:23:17.172 INFO:tasks.workunit.client.0.trial116.stderr:erasure-code-profile myprofile does not exist 2026-02-12T18:23:17.180 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-12T18:23:17.181 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-12T18:23:17.181 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:23:17.181 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-12T18:23:17.181 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:23:17.181 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:23:17.182 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:23:17.183 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:23:17.183 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:23:17.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:23:17.291 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:23:17.292 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:23:17.293 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:23:17.294 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:23:17.294 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:23:17.295 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:23:17.295 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:23:17.295 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:23:17.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:23:17.296 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:23:17.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:23:17.298 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:23:17.298 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-12T18:23:17.315 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:23:17.315 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:23:17.315 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:23:17.315 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76821 2026-02-12T18:23:17.317 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:23:17.317 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:23:17.317 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-12T18:23:17.317 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-12T18:23:17.317 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-12T18:23:17.317 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-12T18:23:17.317 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-12T18:23:17.317 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:23:17.317 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-12T18:23:17.317 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:23:17.317 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:23:17.317 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:23:17.317 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:23:17.318 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:23:17.318 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:23:17.319 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:23:17.319 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:23:17.320 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:23:17.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:23:17.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:23:17.321 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:23:17.322 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:23:17.322 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:23:17.322 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:23:17.322 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:23:17.323 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:23:17.324 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:23:17.324 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-12T18:23:17.325 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:23:17.325 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:23:17.326 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:23:17.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76821 2026-02-12T18:23:17.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:23:17.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:23:17.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-12T18:23:17.328 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T18:23:17.328 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:23:17.328 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:23:17.328 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76821 2026-02-12T18:23:17.329 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T18:23:17.330 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T18:23:17.330 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T18:23:17.330 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-12T18:23:17.330 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-12T18:23:17.330 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-12T18:23:17.330 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T18:23:17.330 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T18:23:17.330 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T18:23:17.330 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-12T18:23:17.330 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-12T18:23:17.348 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T18:23:17.348 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:23:17.348 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:23:17.348 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:23:17.348 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:23:17.348 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:23:17.348 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:23:17.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T18:23:17.372 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T18:23:17.373 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T18:23:17.373 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:23:17.373 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:23:17.373 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T18:23:17.373 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T18:23:17.373 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:23:17.373 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:23:17.373 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:23:17.374 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:23:17.374 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:23:17.374 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:23:17.374 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:23:17.374 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:23:17.374 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76821/ceph-mon.a.asok config get fsid 2026-02-12T18:23:17.417 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T18:23:17.417 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:23:17.417 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:23:17.417 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T18:23:17.417 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T18:23:17.418 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:23:17.418 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:23:17.418 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:23:17.418 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:23:17.419 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:23:17.419 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:23:17.419 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:23:17.419 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:23:17.419 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76821/ceph-mon.a.asok config get mon_host 2026-02-12T18:23:17.460 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-12T18:23:17.460 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-12T18:23:17.460 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T18:23:17.460 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T18:23:17.460 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T18:23:17.460 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-12T18:23:17.460 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T18:23:17.559 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T18:23:17.560 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:23:17.560 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:23:17.560 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:23:17.560 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:23:17.560 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:23:17.560 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:23:17.561 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:23:17.561 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:23:17.576 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-12T18:23:17.577 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T18:23:17.674 INFO:tasks.workunit.client.0.trial116.stderr:pool 'rbd' already exists 2026-02-12T18:23:17.680 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:23:18.682 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-12T18:23:18.683 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:23:18.683 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:23:18.683 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:23:18.683 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:23:18.683 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:23:18.683 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:23:18.683 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-12T18:23:18.683 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-mon.a.asok log flush 2026-02-12T18:23:18.722 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:23:18.726 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-12T18:23:18.727 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:23:17.361+0000 7fef90e20c00 0 load: jerasure load: lrc 2026-02-12T18:23:18.728 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_bad_size_shard_1 td/test-erasure-eio 2026-02-12T18:23:18.728 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:336: TEST_rados_get_bad_size_shard_1: local dir=td/test-erasure-eio 2026-02-12T18:23:18.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:337: TEST_rados_get_bad_size_shard_1: setup_osds 4 2026-02-12T18:23:18.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-12T18:23:18.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-12T18:23:18.729 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-12T18:23:18.729 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-12T18:23:18.734 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:23:18.734 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-12T18:23:18.734 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:23:18.734 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:23:18.734 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T18:23:18.735 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:23:18.735 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-12T18:23:18.735 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:23:18.735 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:23:18.735 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:23:18.735 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:23:18.735 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-12T18:23:18.735 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-12T18:23:18.735 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:23:18.735 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:23:18.735 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:23:18.735 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:23:18.736 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:23:18.736 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:23:18.736 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:23:18.736 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:23:18.736 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:23:18.736 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:23:18.736 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:23:18.736 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:23:18.736 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:23:18.736 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:23:18.736 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:23:18.736 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:23:18.737 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:23:18.737 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:23:18.737 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:23:18.737 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:23:18.737 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:23:18.737 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:23:18.737 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-12T18:23:18.737 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:23:18.737 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c50f675b-8add-48cc-83a3-c0fd7934174a 2026-02-12T18:23:18.737 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 c50f675b-8add-48cc-83a3-c0fd7934174a' 2026-02-12T18:23:18.738 INFO:tasks.workunit.client.0.trial116.stdout:add osd0 c50f675b-8add-48cc-83a3-c0fd7934174a 2026-02-12T18:23:18.738 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:23:18.743 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCWGo5p2mMZLBAAnzVupKURcRFVO8d/7D7rmA== 2026-02-12T18:23:18.744 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCWGo5p2mMZLBAAnzVupKURcRFVO8d/7D7rmA=="}' 2026-02-12T18:23:18.744 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c50f675b-8add-48cc-83a3-c0fd7934174a -i td/test-erasure-eio/0/new.json 2026-02-12T18:23:18.847 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:23:18.853 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-12T18:23:18.854 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCWGo5p2mMZLBAAnzVupKURcRFVO8d/7D7rmA== --osd-uuid c50f675b-8add-48cc-83a3-c0fd7934174a 2026-02-12T18:23:18.869 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:18.866+0000 7f94af462780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:18.871 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:18.867+0000 7f94af462780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:18.871 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:18.868+0000 7f94af462780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:18.872 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:18.868+0000 7f94af462780 -1 bdev(0x3ddcc00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-12T18:23:18.872 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:18.868+0000 7f94af462780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-12T18:23:19.305 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-12T18:23:19.305 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:23:19.306 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T18:23:19.306 INFO:tasks.workunit.client.0.trial116.stdout:adding osd0 key to auth repository 2026-02-12T18:23:19.306 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:23:19.410 INFO:tasks.workunit.client.0.trial116.stdout:start osd.0 2026-02-12T18:23:19.412 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T18:23:19.412 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:23:19.412 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:23:19.412 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:23:19.412 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:23:19.426 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:19.422+0000 7f7b1825b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:19.427 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:19.423+0000 7f7b1825b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:19.428 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:19.424+0000 7f7b1825b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:19.509 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T18:23:19.510 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:23:19.510 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T18:23:19.510 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:23:19.510 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:23:19.510 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:23:19.510 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:23:19.510 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:23:19.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:23:19.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:23:19.609 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:23:19.648 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:19.645+0000 7f7b1825b780 -1 Falling back to public interface 2026-02-12T18:23:19.912 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:19.908+0000 7f7b1825b780 -1 osd.0 0 log_to_monitors true 2026-02-12T18:23:20.610 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:23:20.610 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:23:20.611 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:23:20.611 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:23:20.611 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:23:20.611 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:23:20.843 INFO:tasks.workunit.client.0.trial116.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1028218405,v1:127.0.0.1:6803/1028218405] [v2:127.0.0.1:6804/1028218405,v1:127.0.0.1:6805/1028218405] exists,up c50f675b-8add-48cc-83a3-c0fd7934174a 2026-02-12T18:23:20.848 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:23:20.848 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:23:20.848 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:23:20.848 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:23:20.848 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-12T18:23:20.848 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:23:20.849 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:23:20.849 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-12T18:23:20.849 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:23:20.849 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-12T18:23:20.849 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:23:20.849 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:23:20.849 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:23:20.849 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:23:20.849 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-12T18:23:20.849 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-12T18:23:20.849 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:23:20.849 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:23:20.850 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:23:20.850 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:23:20.850 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:23:20.850 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:23:20.850 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:23:20.850 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:23:20.850 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:23:20.850 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:23:20.850 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:23:20.850 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:23:20.850 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:23:20.850 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:23:20.851 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:23:20.851 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:23:20.851 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:23:20.851 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:23:20.851 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:23:20.851 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:23:20.851 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:23:20.851 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:23:20.851 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-12T18:23:20.851 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:23:20.851 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=bf1f6743-0a0a-4477-a288-553d68123a01 2026-02-12T18:23:20.851 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 bf1f6743-0a0a-4477-a288-553d68123a01' 2026-02-12T18:23:20.852 INFO:tasks.workunit.client.0.trial116.stdout:add osd1 bf1f6743-0a0a-4477-a288-553d68123a01 2026-02-12T18:23:20.852 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:23:20.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCYGo5pXgTwMhAAGJ9OrrX15+N9uBAvfrJEbw== 2026-02-12T18:23:20.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCYGo5pXgTwMhAAGJ9OrrX15+N9uBAvfrJEbw=="}' 2026-02-12T18:23:20.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new bf1f6743-0a0a-4477-a288-553d68123a01 -i td/test-erasure-eio/1/new.json 2026-02-12T18:23:21.088 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:23:21.096 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-12T18:23:21.097 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCYGo5pXgTwMhAAGJ9OrrX15+N9uBAvfrJEbw== --osd-uuid bf1f6743-0a0a-4477-a288-553d68123a01 2026-02-12T18:23:21.113 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:21.109+0000 7f7bf6a22780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:21.114 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:21.110+0000 7f7bf6a22780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:21.115 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:21.111+0000 7f7bf6a22780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:21.116 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:21.112+0000 7f7bf6a22780 -1 bdev(0x2c6cc00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-12T18:23:21.116 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:21.112+0000 7f7bf6a22780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-12T18:23:21.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-12T18:23:21.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:23:21.445 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-12T18:23:21.445 INFO:tasks.workunit.client.0.trial116.stdout:adding osd1 key to auth repository 2026-02-12T18:23:21.446 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:23:21.772 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-12T18:23:21.772 INFO:tasks.workunit.client.0.trial116.stdout:start osd.1 2026-02-12T18:23:21.772 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:23:21.773 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:23:21.773 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:23:21.773 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:23:21.787 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:21.783+0000 7fdd9c839780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:21.788 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:21.784+0000 7fdd9c839780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:21.789 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:21.785+0000 7fdd9c839780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:21.914 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:21.910+0000 7fdd9c839780 -1 Falling back to public interface 2026-02-12T18:23:21.997 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-12T18:23:21.997 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:23:21.997 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T18:23:21.997 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:23:21.997 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:23:21.997 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:23:21.997 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:23:21.998 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:23:21.998 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:23:21.998 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:23:22.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:23:22.224 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:22.221+0000 7fdd9c839780 -1 osd.1 0 log_to_monitors true 2026-02-12T18:23:23.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:23:23.225 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:23:23.225 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:23:23.226 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:23:23.226 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:23:23.226 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:23:23.453 INFO:tasks.workunit.client.0.trial116.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3986571987,v1:127.0.0.1:6811/3986571987] [v2:127.0.0.1:6812/3986571987,v1:127.0.0.1:6813/3986571987] exists,up bf1f6743-0a0a-4477-a288-553d68123a01 2026-02-12T18:23:23.457 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:23:23.457 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:23:23.457 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:23:23.457 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:23:23.458 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-12T18:23:23.458 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:23:23.458 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:23:23.458 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-12T18:23:23.458 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:23:23.458 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-12T18:23:23.458 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:23:23.458 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:23:23.458 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:23:23.458 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:23:23.459 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-12T18:23:23.459 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-12T18:23:23.459 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:23:23.459 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:23:23.459 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:23:23.459 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:23:23.459 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:23:23.459 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:23:23.459 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:23:23.459 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:23:23.459 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:23:23.460 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:23:23.460 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:23:23.460 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:23:23.460 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:23:23.460 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:23:23.460 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:23:23.460 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:23:23.460 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:23:23.460 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:23:23.460 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:23:23.460 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:23:23.461 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:23:23.461 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:23:23.461 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-12T18:23:23.461 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:23:23.461 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0cd3a3c8-cfaa-4d5e-ad0c-fd4c14cc223c 2026-02-12T18:23:23.461 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 0cd3a3c8-cfaa-4d5e-ad0c-fd4c14cc223c' 2026-02-12T18:23:23.461 INFO:tasks.workunit.client.0.trial116.stdout:add osd2 0cd3a3c8-cfaa-4d5e-ad0c-fd4c14cc223c 2026-02-12T18:23:23.462 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:23:23.466 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCbGo5p1BGhGxAAAx+bBIfAweOm7ew4N0ALRA== 2026-02-12T18:23:23.467 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCbGo5p1BGhGxAAAx+bBIfAweOm7ew4N0ALRA=="}' 2026-02-12T18:23:23.467 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0cd3a3c8-cfaa-4d5e-ad0c-fd4c14cc223c -i td/test-erasure-eio/2/new.json 2026-02-12T18:23:23.697 INFO:tasks.workunit.client.0.trial116.stdout:2 2026-02-12T18:23:23.705 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-12T18:23:23.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCbGo5p1BGhGxAAAx+bBIfAweOm7ew4N0ALRA== --osd-uuid 0cd3a3c8-cfaa-4d5e-ad0c-fd4c14cc223c 2026-02-12T18:23:23.722 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:23.718+0000 7f9210c67780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:23.724 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:23.720+0000 7f9210c67780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:23.725 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:23.721+0000 7f9210c67780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:23.725 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:23.721+0000 7f9210c67780 -1 bdev(0x331cc00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-12T18:23:23.725 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:23.721+0000 7f9210c67780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-12T18:23:24.035 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-12T18:23:24.035 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:23:24.035 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-12T18:23:24.036 INFO:tasks.workunit.client.0.trial116.stdout:adding osd2 key to auth repository 2026-02-12T18:23:24.036 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:23:24.355 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-12T18:23:24.356 INFO:tasks.workunit.client.0.trial116.stdout:start osd.2 2026-02-12T18:23:24.357 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:23:24.357 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:23:24.357 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:23:24.357 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:23:24.370 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:24.366+0000 7efee3a55780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:24.371 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:24.368+0000 7efee3a55780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:24.372 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:24.368+0000 7efee3a55780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:24.542 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:24.538+0000 7efee3a55780 -1 Falling back to public interface 2026-02-12T18:23:24.583 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-12T18:23:24.584 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:23:24.584 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T18:23:24.584 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:23:24.584 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:23:24.584 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:23:24.584 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:23:24.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:23:24.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:23:24.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:23:24.818 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:23:24.923 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:24.919+0000 7efee3a55780 -1 osd.2 0 log_to_monitors true 2026-02-12T18:23:25.820 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:23:25.820 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:23:25.820 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:23:25.821 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:23:25.821 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:23:25.821 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:23:26.047 INFO:tasks.workunit.client.0.trial116.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/73784054,v1:127.0.0.1:6819/73784054] [v2:127.0.0.1:6820/73784054,v1:127.0.0.1:6821/73784054] exists,up 0cd3a3c8-cfaa-4d5e-ad0c-fd4c14cc223c 2026-02-12T18:23:26.051 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:23:26.051 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:23:26.051 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:23:26.051 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:23:26.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-12T18:23:26.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:23:26.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:23:26.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-12T18:23:26.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:23:26.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-12T18:23:26.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:23:26.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:23:26.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:23:26.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:23:26.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-12T18:23:26.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-12T18:23:26.053 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:23:26.053 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:23:26.053 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:23:26.053 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:23:26.053 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:23:26.053 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:23:26.053 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:23:26.053 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:23:26.053 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:23:26.053 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:23:26.053 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:23:26.053 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:23:26.054 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:23:26.054 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:23:26.054 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:23:26.054 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:23:26.054 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:23:26.054 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:23:26.054 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:23:26.054 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:23:26.054 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:23:26.054 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:23:26.054 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-12T18:23:26.054 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:23:26.054 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=878a81b5-5c7c-42bb-9139-e269a4aa40f2 2026-02-12T18:23:26.055 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 878a81b5-5c7c-42bb-9139-e269a4aa40f2' 2026-02-12T18:23:26.055 INFO:tasks.workunit.client.0.trial116.stdout:add osd3 878a81b5-5c7c-42bb-9139-e269a4aa40f2 2026-02-12T18:23:26.055 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:23:26.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCeGo5pLZV0AxAAZaK28ly55O4QJYrzJC8gAQ== 2026-02-12T18:23:26.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCeGo5pLZV0AxAAZaK28ly55O4QJYrzJC8gAQ=="}' 2026-02-12T18:23:26.061 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 878a81b5-5c7c-42bb-9139-e269a4aa40f2 -i td/test-erasure-eio/3/new.json 2026-02-12T18:23:26.294 INFO:tasks.workunit.client.0.trial116.stdout:3 2026-02-12T18:23:26.303 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-12T18:23:26.304 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCeGo5pLZV0AxAAZaK28ly55O4QJYrzJC8gAQ== --osd-uuid 878a81b5-5c7c-42bb-9139-e269a4aa40f2 2026-02-12T18:23:26.320 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:26.316+0000 7fd9b4a0b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:26.321 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:26.317+0000 7fd9b4a0b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:26.322 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:26.318+0000 7fd9b4a0b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:26.322 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:26.319+0000 7fd9b4a0b780 -1 bdev(0x4214c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-12T18:23:26.323 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:26.319+0000 7fd9b4a0b780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-12T18:23:26.671 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-12T18:23:26.671 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:23:26.672 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-12T18:23:26.672 INFO:tasks.workunit.client.0.trial116.stdout:adding osd3 key to auth repository 2026-02-12T18:23:26.672 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:23:27.001 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-12T18:23:27.001 INFO:tasks.workunit.client.0.trial116.stdout:start osd.3 2026-02-12T18:23:27.002 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:23:27.002 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:23:27.002 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:23:27.002 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:23:27.015 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:27.012+0000 7f566d212780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:27.017 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:27.013+0000 7f566d212780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:27.018 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:27.014+0000 7f566d212780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:27.142 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:27.138+0000 7f566d212780 -1 Falling back to public interface 2026-02-12T18:23:27.228 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-12T18:23:27.228 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:23:27.228 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-12T18:23:27.228 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:23:27.228 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:23:27.228 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:23:27.228 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:23:27.228 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:23:27.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:23:27.229 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:23:27.425 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:27.422+0000 7f566d212780 -1 osd.3 0 log_to_monitors true 2026-02-12T18:23:27.457 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:23:28.458 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:23:28.458 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:23:28.459 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:23:28.459 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:23:28.459 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:23:28.460 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:23:28.685 INFO:tasks.workunit.client.0.trial116.stdout:osd.3 up in weight 1 up_from 25 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3899340392,v1:127.0.0.1:6827/3899340392] [v2:127.0.0.1:6828/3899340392,v1:127.0.0.1:6829/3899340392] exists,up 878a81b5-5c7c-42bb-9139-e269a4aa40f2 2026-02-12T18:23:28.685 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:23:28.685 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:23:28.685 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:23:28.685 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-12T18:23:28.685 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T18:23:28.685 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T18:23:28.686 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:23:28.686 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:23:28.686 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:23:28.686 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.0.asok 2026-02-12T18:23:28.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-12T18:23:28.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-osd.0.asok log flush 2026-02-12T18:23:28.730 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:23:28.733 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-12T18:23:28.736 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:23:19.647+0000 7f7b1825b780 0 load: jerasure load: lrc 2026-02-12T18:23:28.736 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:339: TEST_rados_get_bad_size_shard_1: local poolname=pool-jerasure 2026-02-12T18:23:28.736 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:340: TEST_rados_get_bad_size_shard_1: create_erasure_coded_pool pool-jerasure 2 1 2026-02-12T18:23:28.736 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-12T18:23:28.736 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-12T18:23:28.736 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-12T18:23:28.736 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-12T18:23:28.736 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-12T18:23:28.736 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-12T18:23:28.736 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-12T18:23:29.023 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-12T18:23:29.023 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-12T18:23:29.395 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-jerasure' already exists 2026-02-12T18:23:29.402 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:23:30.404 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-12T18:23:30.404 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:23:30.404 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:23:30.404 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:23:30.404 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:23:30.405 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:23:30.405 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:23:30.405 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:23:30.405 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:23:30.405 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:23:30.492 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:23:30.492 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:23:30.492 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:23:30.492 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:23:30.492 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:23:30.492 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:23:30.718 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:23:30.719 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:23:30.719 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:23:30.719 INFO:tasks.workunit.client.0.trial116.stderr:3' 2026-02-12T18:23:30.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:23:30.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:30.719 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:23:30.781 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803780 2026-02-12T18:23:30.781 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803780 2026-02-12T18:23:30.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780' 2026-02-12T18:23:30.782 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:30.782 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:23:30.845 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607557 2026-02-12T18:23:30.845 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607557 2026-02-12T18:23:30.845 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-51539607557' 2026-02-12T18:23:30.846 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:30.846 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:23:30.907 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411331 2026-02-12T18:23:30.908 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411331 2026-02-12T18:23:30.908 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-51539607557 2-77309411331' 2026-02-12T18:23:30.908 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:30.908 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:23:30.971 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182404 2026-02-12T18:23:30.972 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182404 2026-02-12T18:23:30.972 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-51539607557 2-77309411331 3-107374182404' 2026-02-12T18:23:30.972 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:23:30.973 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803780 2026-02-12T18:23:30.973 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:23:30.973 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:23:30.974 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803780 2026-02-12T18:23:30.974 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:23:30.975 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803780 2026-02-12T18:23:30.975 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803780' 2026-02-12T18:23:30.975 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.0 seq 25769803780 2026-02-12T18:23:30.976 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:23:31.207 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803778 -lt 25769803780 2026-02-12T18:23:31.207 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:23:32.208 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:23:32.209 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:23:32.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803780 2026-02-12T18:23:32.434 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:23:32.434 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607557 2026-02-12T18:23:32.434 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:23:32.435 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:23:32.436 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607557 2026-02-12T18:23:32.436 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:23:32.437 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607557 2026-02-12T18:23:32.437 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607557' 2026-02-12T18:23:32.438 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.1 seq 51539607557 2026-02-12T18:23:32.438 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:23:32.667 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607557 -lt 51539607557 2026-02-12T18:23:32.667 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:23:32.668 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411331 2026-02-12T18:23:32.668 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:23:32.669 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:23:32.669 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411331 2026-02-12T18:23:32.670 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:23:32.671 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411331 2026-02-12T18:23:32.671 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411331' 2026-02-12T18:23:32.671 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.2 seq 77309411331 2026-02-12T18:23:32.671 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:23:32.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411332 -lt 77309411331 2026-02-12T18:23:32.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:23:32.907 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182404 2026-02-12T18:23:32.907 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:23:32.907 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:23:32.908 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182404 2026-02-12T18:23:32.908 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:23:32.909 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182404 2026-02-12T18:23:32.909 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182404' 2026-02-12T18:23:32.910 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.3 seq 107374182404 2026-02-12T18:23:32.910 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:23:33.138 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182404 -lt 107374182404 2026-02-12T18:23:33.139 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:23:33.139 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:23:33.139 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:23:33.457 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:23:33.457 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:23:33.457 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:23:33.457 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:23:33.457 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:23:33.458 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:23:33.458 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:23:33.458 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:23:33.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:23:33.686 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:23:33.686 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:23:33.687 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:23:34.002 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:23:34.002 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:23:34.002 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:23:34.003 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:342: TEST_rados_get_bad_size_shard_1: local shard_id=1 2026-02-12T18:23:34.003 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:343: TEST_rados_get_bad_size_shard_1: rados_get_data_bad_size td/test-erasure-eio 1 10 2026-02-12T18:23:34.003 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-12T18:23:34.003 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-12T18:23:34.003 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=1 2026-02-12T18:23:34.003 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-12T18:23:34.003 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=10 2026-02-12T18:23:34.003 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-12T18:23:34.003 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-12T18:23:34.003 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-12T18:23:34.003 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-76821-1-10 2026-02-12T18:23:34.003 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-76821-1-10 2026-02-12T18:23:34.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-12T18:23:34.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-12T18:23:34.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-76821-1-10 2026-02-12T18:23:34.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:23:34.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-12T18:23:34.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:23:34.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-12T18:23:34.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:23:34.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-12T18:23:34.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:23:34.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-12T18:23:34.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-76821-1-10 td/test-erasure-eio/ORIGINAL 2026-02-12T18:23:34.019 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-76821-1-10 td/test-erasure-eio 1 10 set 2026-02-12T18:23:34.019 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-76821-1-10 2026-02-12T18:23:34.019 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-12T18:23:34.019 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-12T18:23:34.019 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-12T18:23:34.019 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-12T18:23:34.019 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-12T18:23:34.019 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-12T18:23:34.020 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-12T18:23:34.020 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-12T18:23:34.020 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-12T18:23:34.020 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-76821-1-10 2026-02-12T18:23:34.020 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:23:34.020 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-76821-1-10 2026-02-12T18:23:34.020 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-76821-1-10 2026-02-12T18:23:34.020 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:23:34.248 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:23:34.249 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:23:34.249 INFO:tasks.workunit.client.0.trial116.stderr:2' 2026-02-12T18:23:34.249 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-12T18:23:34.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-12T18:23:34.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-12T18:23:34.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-12T18:23:34.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-12T18:23:34.530 INFO:tasks.workunit.client.0.trial116.stderr:noout is set 2026-02-12T18:23:34.540 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-12T18:23:34.540 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-12T18:23:34.540 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-12T18:23:34.542 INFO:tasks.workunit.client.0.trial116.stderr:1+0 records in 2026-02-12T18:23:34.542 INFO:tasks.workunit.client.0.trial116.stderr:1+0 records out 2026-02-12T18:23:34.543 INFO:tasks.workunit.client.0.trial116.stderr:10 bytes copied, 3.6588e-05 s, 273 kB/s 2026-02-12T18:23:34.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-76821-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:23:34.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-12T18:23:34.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-12T18:23:34.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-12T18:23:34.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-12T18:23:34.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-76821-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:23:34.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-12T18:23:34.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-12T18:23:34.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-12T18:23:34.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-12T18:23:34.544 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-12T18:23:34.544 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:23:34.544 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:23:34.544 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:23:34.544 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:23:34.544 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:23:34.849 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:23:34.849 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-76821-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:23:34.849 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-12T18:23:34.849 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-12T18:23:34.849 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-12T18:23:34.850 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-12T18:23:34.850 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-12T18:23:34.850 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-76821-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:23:34.992 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-12T18:23:34.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-12T18:23:34.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T18:23:34.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-12T18:23:34.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T18:23:34.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-12T18:23:34.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:23:34.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:23:34.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:23:34.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:23:34.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-12T18:23:34.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-12T18:23:34.994 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T18:23:34.994 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T18:23:34.994 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:23:34.994 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T18:23:34.994 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:23:34.994 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:23:34.994 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:23:34.994 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:23:34.994 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:23:34.994 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:23:34.994 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:23:34.994 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:23:34.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:23:34.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:23:34.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:23:34.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:23:34.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:23:34.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:23:34.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T18:23:34.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-12T18:23:34.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-12T18:23:34.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-12T18:23:34.995 INFO:tasks.workunit.client.0.trial116.stderr:start osd.1 2026-02-12T18:23:34.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:23:34.996 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-12T18:23:34.996 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-12T18:23:34.996 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T18:23:34.997 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T18:23:34.997 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T18:23:35.006 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:35.002+0000 7f2b2f813780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:35.008 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:35.004+0000 7f2b2f813780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:35.008 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:35.005+0000 7f2b2f813780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:35.140 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:35.137+0000 7f2b2f813780 -1 Falling back to public interface 2026-02-12T18:23:35.219 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-12T18:23:35.219 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:23:35.220 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T18:23:35.220 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:23:35.220 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:23:35.220 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:23:35.220 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:23:35.220 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:23:35.220 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:23:35.220 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:23:35.446 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:23:35.460 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:35.456+0000 7f2b2f813780 -1 osd.1 33 log_to_monitors true 2026-02-12T18:23:36.448 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:23:36.448 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:23:36.448 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:23:36.448 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:23:36.448 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:23:36.448 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:23:36.673 INFO:tasks.workunit.client.0.trial116.stderr:osd.1 up in weight 1 up_from 39 up_thru 39 down_at 36 last_clean_interval [12,33) [v2:127.0.0.1:6810/4056996757,v1:127.0.0.1:6811/4056996757] [v2:127.0.0.1:6812/4056996757,v1:127.0.0.1:6813/4056996757] exists,up bf1f6743-0a0a-4477-a288-553d68123a01 2026-02-12T18:23:36.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:23:36.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:23:36.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:23:36.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-12T18:23:36.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:23:36.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:23:36.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:23:36.674 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:23:36.675 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:23:36.675 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:23:36.675 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:23:36.675 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:23:36.676 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:23:36.760 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:23:36.760 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:23:36.760 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:23:36.760 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:23:36.760 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:23:36.760 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:23:36.994 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:23:36.994 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:23:36.994 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:23:36.994 INFO:tasks.workunit.client.0.trial116.stderr:3' 2026-02-12T18:23:36.994 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:23:36.995 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:36.995 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:23:37.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803784 2026-02-12T18:23:37.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803784 2026-02-12T18:23:37.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784' 2026-02-12T18:23:37.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:37.059 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:23:37.126 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724547 2026-02-12T18:23:37.126 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724547 2026-02-12T18:23:37.126 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-167503724547' 2026-02-12T18:23:37.126 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:37.126 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:23:37.190 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411335 2026-02-12T18:23:37.190 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411335 2026-02-12T18:23:37.190 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-167503724547 2-77309411335' 2026-02-12T18:23:37.190 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:37.190 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:23:37.257 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182407 2026-02-12T18:23:37.257 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182407 2026-02-12T18:23:37.257 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-167503724547 2-77309411335 3-107374182407' 2026-02-12T18:23:37.257 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:23:37.258 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803784 2026-02-12T18:23:37.258 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:23:37.258 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:23:37.259 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803784 2026-02-12T18:23:37.259 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:23:37.260 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803784 2026-02-12T18:23:37.260 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-12T18:23:37.260 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.0 seq 25769803784 2026-02-12T18:23:37.261 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:23:37.485 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803784 2026-02-12T18:23:37.486 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:23:38.486 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:23:38.487 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:23:38.713 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-12T18:23:38.713 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:23:38.714 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-167503724547 2026-02-12T18:23:38.714 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:23:38.715 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:23:38.715 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-167503724547 2026-02-12T18:23:38.715 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:23:38.716 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=167503724547 2026-02-12T18:23:38.717 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 167503724547' 2026-02-12T18:23:38.717 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.1 seq 167503724547 2026-02-12T18:23:38.717 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:23:38.946 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 167503724547 -lt 167503724547 2026-02-12T18:23:38.946 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:23:38.947 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411335 2026-02-12T18:23:38.947 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:23:38.948 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:23:38.948 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411335 2026-02-12T18:23:38.949 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:23:38.949 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411335 2026-02-12T18:23:38.950 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411335' 2026-02-12T18:23:38.950 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.2 seq 77309411335 2026-02-12T18:23:38.950 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:23:39.175 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411335 -lt 77309411335 2026-02-12T18:23:39.176 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:23:39.176 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182407 2026-02-12T18:23:39.176 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:23:39.177 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:23:39.178 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182407 2026-02-12T18:23:39.178 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:23:39.179 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182407 2026-02-12T18:23:39.179 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182407' 2026-02-12T18:23:39.179 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.3 seq 107374182407 2026-02-12T18:23:39.179 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:23:39.405 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182407 -lt 107374182407 2026-02-12T18:23:39.406 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:23:39.406 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:23:39.406 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:23:39.726 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:23:39.726 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:23:39.726 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:23:39.726 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:23:39.726 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:23:39.726 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:23:39.727 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:23:39.727 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:23:39.948 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:23:39.949 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:23:39.949 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:23:39.950 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:23:40.268 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:23:40.268 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:23:40.268 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:23:40.268 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-12T18:23:40.269 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-12T18:23:40.583 INFO:tasks.workunit.client.0.trial116.stderr:noout is unset 2026-02-12T18:23:40.594 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-76821-1-10 2026-02-12T18:23:40.594 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-12T18:23:40.594 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-12T18:23:40.594 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-76821-1-10 2026-02-12T18:23:40.594 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-12T18:23:40.594 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-12T18:23:40.595 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-76821-1-10 td/test-erasure-eio/COPY 2026-02-12T18:23:40.612 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-12T18:23:40.613 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-12T18:23:40.614 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 1 + 1 2026-02-12T18:23:40.616 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=2 2026-02-12T18:23:40.616 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-76821-1-10 td/test-erasure-eio 2 10 set 2026-02-12T18:23:40.617 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-76821-1-10 2026-02-12T18:23:40.617 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-12T18:23:40.617 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-12T18:23:40.617 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-12T18:23:40.617 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=2 2026-02-12T18:23:40.617 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-12T18:23:40.617 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-12T18:23:40.617 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-12T18:23:40.617 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-12T18:23:40.617 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-12T18:23:40.617 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-76821-1-10 2026-02-12T18:23:40.618 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:23:40.618 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-76821-1-10 2026-02-12T18:23:40.618 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-76821-1-10 2026-02-12T18:23:40.618 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:23:40.850 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:23:40.850 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:23:40.850 INFO:tasks.workunit.client.0.trial116.stderr:2' 2026-02-12T18:23:40.850 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-12T18:23:40.850 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-12T18:23:40.850 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-12T18:23:40.850 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=2 2026-02-12T18:23:40.850 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-12T18:23:41.160 INFO:tasks.workunit.client.0.trial116.stderr:noout is set 2026-02-12T18:23:41.170 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-12T18:23:41.170 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-12T18:23:41.170 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-12T18:23:41.172 INFO:tasks.workunit.client.0.trial116.stderr:1+0 records in 2026-02-12T18:23:41.172 INFO:tasks.workunit.client.0.trial116.stderr:1+0 records out 2026-02-12T18:23:41.172 INFO:tasks.workunit.client.0.trial116.stderr:10 bytes copied, 3.8331e-05 s, 261 kB/s 2026-02-12T18:23:41.172 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-76821-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:23:41.172 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-12T18:23:41.173 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-12T18:23:41.173 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-12T18:23:41.173 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-12T18:23:41.173 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-76821-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:23:41.173 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-12T18:23:41.173 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-12T18:23:41.173 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-12T18:23:41.173 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-12T18:23:41.173 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-12T18:23:41.173 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:23:41.173 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:23:41.173 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:23:41.174 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:23:41.174 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:23:41.478 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:23:41.478 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-76821-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:23:41.478 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-12T18:23:41.478 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-12T18:23:41.479 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-12T18:23:41.479 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-12T18:23:41.479 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-12T18:23:41.479 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-76821-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:23:41.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-12T18:23:41.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-12T18:23:41.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T18:23:41.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-12T18:23:41.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T18:23:41.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-12T18:23:41.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:23:41.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:23:41.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:23:41.628 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:23:41.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-12T18:23:41.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-12T18:23:41.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T18:23:41.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T18:23:41.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:23:41.629 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T18:23:41.629 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:23:41.629 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:23:41.629 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:23:41.629 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:23:41.629 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:23:41.629 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:23:41.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:23:41.630 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:23:41.630 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:23:41.630 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:23:41.630 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:23:41.630 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:23:41.630 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:23:41.630 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:23:41.630 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T18:23:41.630 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-12T18:23:41.630 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-12T18:23:41.630 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-12T18:23:41.630 INFO:tasks.workunit.client.0.trial116.stderr:start osd.2 2026-02-12T18:23:41.630 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:23:41.631 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-12T18:23:41.631 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-12T18:23:41.631 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T18:23:41.632 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T18:23:41.632 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T18:23:41.641 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:41.637+0000 7f45f9036780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:41.643 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:41.639+0000 7f45f9036780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:41.643 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:41.640+0000 7f45f9036780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:41.776 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:41.773+0000 7f45f9036780 -1 Falling back to public interface 2026-02-12T18:23:41.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-12T18:23:41.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:23:41.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T18:23:41.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:23:41.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:23:41.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:23:41.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:23:41.858 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:23:41.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:23:41.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:23:42.084 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:23:42.136 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:42.133+0000 7f45f9036780 -1 osd.2 42 log_to_monitors true 2026-02-12T18:23:43.086 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:23:43.086 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:23:43.086 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:23:43.086 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:23:43.086 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:23:43.086 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:23:43.314 INFO:tasks.workunit.client.0.trial116.stderr:osd.2 up in weight 1 up_from 48 up_thru 36 down_at 45 last_clean_interval [18,42) [v2:127.0.0.1:6818/2656949107,v1:127.0.0.1:6819/2656949107] [v2:127.0.0.1:6820/2656949107,v1:127.0.0.1:6821/2656949107] exists,up 0cd3a3c8-cfaa-4d5e-ad0c-fd4c14cc223c 2026-02-12T18:23:43.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:23:43.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:23:43.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:23:43.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-12T18:23:43.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:23:43.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:23:43.314 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:23:43.314 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:23:43.315 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:23:43.315 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:23:43.315 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:23:43.316 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:23:43.316 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:23:43.400 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:23:43.400 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:23:43.400 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:23:43.400 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:23:43.400 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:23:43.400 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:23:43.626 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:23:43.626 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:23:43.626 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:23:43.626 INFO:tasks.workunit.client.0.trial116.stderr:3' 2026-02-12T18:23:43.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:23:43.627 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:43.627 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:23:43.693 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803787 2026-02-12T18:23:43.693 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803787 2026-02-12T18:23:43.694 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787' 2026-02-12T18:23:43.694 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:43.694 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:23:43.760 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724550 2026-02-12T18:23:43.760 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724550 2026-02-12T18:23:43.760 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-167503724550' 2026-02-12T18:23:43.760 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:43.760 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:23:43.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=206158430211 2026-02-12T18:23:43.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 206158430211 2026-02-12T18:23:43.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-167503724550 2-206158430211' 2026-02-12T18:23:43.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:43.827 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:23:43.893 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182411 2026-02-12T18:23:43.893 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182411 2026-02-12T18:23:43.894 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-167503724550 2-206158430211 3-107374182411' 2026-02-12T18:23:43.894 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:23:43.894 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803787 2026-02-12T18:23:43.894 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:23:43.894 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:23:43.895 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803787 2026-02-12T18:23:43.895 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:23:43.896 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803787 2026-02-12T18:23:43.896 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803787' 2026-02-12T18:23:43.896 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.0 seq 25769803787 2026-02-12T18:23:43.896 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:23:44.121 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803785 -lt 25769803787 2026-02-12T18:23:44.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:23:45.123 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:23:45.123 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:23:45.350 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803785 -lt 25769803787 2026-02-12T18:23:45.350 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:23:46.351 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-12T18:23:46.352 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:23:46.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803787 -lt 25769803787 2026-02-12T18:23:46.590 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:23:46.590 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-167503724550 2026-02-12T18:23:46.590 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:23:46.591 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:23:46.592 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-167503724550 2026-02-12T18:23:46.592 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:23:46.593 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=167503724550 2026-02-12T18:23:46.593 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 167503724550' 2026-02-12T18:23:46.593 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.1 seq 167503724550 2026-02-12T18:23:46.593 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:23:46.817 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 167503724551 -lt 167503724550 2026-02-12T18:23:46.818 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:23:46.818 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-206158430211 2026-02-12T18:23:46.818 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:23:46.819 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:23:46.819 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-206158430211 2026-02-12T18:23:46.819 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:23:46.821 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=206158430211 2026-02-12T18:23:46.821 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 206158430211' 2026-02-12T18:23:46.821 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.2 seq 206158430211 2026-02-12T18:23:46.821 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:23:47.047 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 206158430211 -lt 206158430211 2026-02-12T18:23:47.048 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:23:47.048 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182411 2026-02-12T18:23:47.049 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:23:47.049 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:23:47.049 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182411 2026-02-12T18:23:47.050 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:23:47.051 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182411 2026-02-12T18:23:47.051 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182411' 2026-02-12T18:23:47.051 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.3 seq 107374182411 2026-02-12T18:23:47.051 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:23:47.276 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182411 -lt 107374182411 2026-02-12T18:23:47.277 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:23:47.277 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:23:47.277 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:23:47.592 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:23:47.592 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:23:47.592 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:23:47.592 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:23:47.592 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:23:47.593 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:23:47.593 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:23:47.593 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:23:47.815 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:23:47.816 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:23:47.816 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:23:47.816 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:23:48.134 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:23:48.134 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:23:48.134 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:23:48.134 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-12T18:23:48.135 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-12T18:23:48.449 INFO:tasks.workunit.client.0.trial116.stderr:noout is unset 2026-02-12T18:23:48.459 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-12T18:23:48.459 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:258: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-76821-1-10 2026-02-12T18:23:48.459 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-12T18:23:48.459 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-12T18:23:48.459 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-76821-1-10 2026-02-12T18:23:48.459 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-12T18:23:48.459 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-12T18:23:48.459 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-76821-1-10 td/test-erasure-eio/COPY 2026-02-12T18:23:48.477 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-12T18:23:48.478 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-12T18:23:48.479 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-12T18:23:48.482 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:344: TEST_rados_get_bad_size_shard_1: rados_get_data_bad_size td/test-erasure-eio 1 0 2026-02-12T18:23:48.482 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-12T18:23:48.482 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-12T18:23:48.482 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=1 2026-02-12T18:23:48.482 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-12T18:23:48.482 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=0 2026-02-12T18:23:48.482 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-12T18:23:48.482 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-12T18:23:48.482 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-12T18:23:48.482 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-76821-1-0 2026-02-12T18:23:48.482 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-76821-1-0 2026-02-12T18:23:48.483 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-12T18:23:48.483 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-12T18:23:48.483 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-76821-1-0 2026-02-12T18:23:48.483 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:23:48.483 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-12T18:23:48.483 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:23:48.483 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-12T18:23:48.483 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:23:48.483 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-12T18:23:48.483 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:23:48.483 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-12T18:23:48.483 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-76821-1-0 td/test-erasure-eio/ORIGINAL 2026-02-12T18:23:48.498 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-76821-1-0 td/test-erasure-eio 1 0 set 2026-02-12T18:23:48.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-76821-1-0 2026-02-12T18:23:48.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-12T18:23:48.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-12T18:23:48.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-12T18:23:48.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-12T18:23:48.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-12T18:23:48.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-12T18:23:48.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-12T18:23:48.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-12T18:23:48.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-12T18:23:48.499 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-76821-1-0 2026-02-12T18:23:48.499 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:23:48.500 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-76821-1-0 2026-02-12T18:23:48.500 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-76821-1-0 2026-02-12T18:23:48.500 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:23:48.726 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:23:48.726 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:23:48.727 INFO:tasks.workunit.client.0.trial116.stderr:2' 2026-02-12T18:23:48.727 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-12T18:23:48.727 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-12T18:23:48.727 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-12T18:23:48.727 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-12T18:23:48.727 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-12T18:23:49.040 INFO:tasks.workunit.client.0.trial116.stderr:noout is set 2026-02-12T18:23:49.051 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-12T18:23:49.051 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-12T18:23:49.051 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-12T18:23:49.053 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-76821-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:23:49.054 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-12T18:23:49.054 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-12T18:23:49.054 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-12T18:23:49.054 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-12T18:23:49.054 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-76821-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:23:49.054 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-12T18:23:49.054 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-12T18:23:49.054 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-12T18:23:49.054 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-12T18:23:49.054 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-12T18:23:49.055 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:23:49.055 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:23:49.055 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:23:49.055 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:23:49.055 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:23:49.359 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:23:49.360 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-76821-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:23:49.360 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-12T18:23:49.360 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-12T18:23:49.360 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-12T18:23:49.360 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-12T18:23:49.360 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-12T18:23:49.360 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-76821-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:23:49.498 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-12T18:23:49.498 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-12T18:23:49.498 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T18:23:49.498 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-12T18:23:49.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T18:23:49.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-12T18:23:49.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:23:49.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:23:49.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:23:49.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:23:49.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-12T18:23:49.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-12T18:23:49.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T18:23:49.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T18:23:49.499 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:23:49.499 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T18:23:49.500 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:23:49.500 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:23:49.500 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:23:49.500 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:23:49.500 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:23:49.500 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:23:49.500 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:23:49.500 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:23:49.500 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:23:49.500 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:23:49.500 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:23:49.500 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:23:49.501 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:23:49.501 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:23:49.501 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T18:23:49.501 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-12T18:23:49.501 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-12T18:23:49.501 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-12T18:23:49.501 INFO:tasks.workunit.client.0.trial116.stderr:start osd.1 2026-02-12T18:23:49.501 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:23:49.501 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-12T18:23:49.502 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-12T18:23:49.502 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T18:23:49.502 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T18:23:49.502 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T18:23:49.513 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:49.509+0000 7f1ac220c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:49.514 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:49.511+0000 7f1ac220c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:49.515 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:49.512+0000 7f1ac220c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:49.664 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:49.662+0000 7f1ac220c780 -1 Falling back to public interface 2026-02-12T18:23:49.729 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-12T18:23:49.730 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:23:49.730 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T18:23:49.730 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:23:49.730 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:23:49.730 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:23:49.730 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:23:49.730 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:23:49.730 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:23:49.730 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:23:49.959 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:23:50.027 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:50.025+0000 7f1ac220c780 -1 osd.1 52 log_to_monitors true 2026-02-12T18:23:50.962 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:23:50.962 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:23:50.962 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:23:50.962 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:23:50.962 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:23:50.962 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:23:51.198 INFO:tasks.workunit.client.0.trial116.stderr:osd.1 up in weight 1 up_from 58 up_thru 58 down_at 55 last_clean_interval [39,52) [v2:127.0.0.1:6810/2811344982,v1:127.0.0.1:6811/2811344982] [v2:127.0.0.1:6812/2811344982,v1:127.0.0.1:6813/2811344982] exists,up bf1f6743-0a0a-4477-a288-553d68123a01 2026-02-12T18:23:51.198 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:23:51.198 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:23:51.198 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:23:51.198 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-12T18:23:51.198 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:23:51.198 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:23:51.199 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:23:51.199 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:23:51.199 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:23:51.200 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:23:51.200 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:23:51.200 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:23:51.200 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:23:51.284 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:23:51.284 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:23:51.284 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:23:51.284 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:23:51.284 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:23:51.284 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:23:51.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:23:51.512 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:23:51.512 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:23:51.512 INFO:tasks.workunit.client.0.trial116.stderr:3' 2026-02-12T18:23:51.512 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:23:51.512 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:51.512 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:23:51.576 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803791 2026-02-12T18:23:51.576 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803791 2026-02-12T18:23:51.576 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791' 2026-02-12T18:23:51.576 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:51.577 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:23:51.642 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=249108103171 2026-02-12T18:23:51.642 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 249108103171 2026-02-12T18:23:51.642 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-249108103171' 2026-02-12T18:23:51.642 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:51.642 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:23:51.707 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=206158430214 2026-02-12T18:23:51.707 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 206158430214 2026-02-12T18:23:51.707 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-249108103171 2-206158430214' 2026-02-12T18:23:51.707 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:51.707 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:23:51.771 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182414 2026-02-12T18:23:51.771 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182414 2026-02-12T18:23:51.771 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803791 1-249108103171 2-206158430214 3-107374182414' 2026-02-12T18:23:51.771 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:23:51.772 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803791 2026-02-12T18:23:51.772 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:23:51.772 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:23:51.773 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803791 2026-02-12T18:23:51.773 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:23:51.774 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803791 2026-02-12T18:23:51.774 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803791' 2026-02-12T18:23:51.774 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.0 seq 25769803791 2026-02-12T18:23:51.775 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:23:52.000 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803789 -lt 25769803791 2026-02-12T18:23:52.000 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:23:53.001 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:23:53.002 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:23:53.233 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803789 -lt 25769803791 2026-02-12T18:23:53.233 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:23:54.233 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-12T18:23:54.234 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:23:54.459 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803791 -lt 25769803791 2026-02-12T18:23:54.459 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:23:54.460 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-249108103171 2026-02-12T18:23:54.460 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:23:54.461 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:23:54.462 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-249108103171 2026-02-12T18:23:54.462 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:23:54.463 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=249108103171 2026-02-12T18:23:54.463 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 249108103171' 2026-02-12T18:23:54.463 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.1 seq 249108103171 2026-02-12T18:23:54.463 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:23:54.688 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 249108103171 -lt 249108103171 2026-02-12T18:23:54.688 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:23:54.688 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-206158430214 2026-02-12T18:23:54.688 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:23:54.689 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:23:54.690 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-206158430214 2026-02-12T18:23:54.690 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:23:54.691 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=206158430214 2026-02-12T18:23:54.691 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 206158430214' 2026-02-12T18:23:54.691 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.2 seq 206158430214 2026-02-12T18:23:54.691 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:23:54.917 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 206158430215 -lt 206158430214 2026-02-12T18:23:54.917 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:23:54.918 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182414 2026-02-12T18:23:54.918 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:23:54.919 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:23:54.919 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182414 2026-02-12T18:23:54.919 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:23:54.920 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182414 2026-02-12T18:23:54.920 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182414' 2026-02-12T18:23:54.920 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.3 seq 107374182414 2026-02-12T18:23:54.921 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:23:55.145 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182414 -lt 107374182414 2026-02-12T18:23:55.146 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:23:55.146 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:23:55.146 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:23:55.463 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:23:55.464 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:23:55.464 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:23:55.464 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:23:55.464 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:23:55.464 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:23:55.464 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:23:55.464 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:23:55.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:23:55.687 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:23:55.687 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:23:55.687 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:23:56.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:23:56.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:23:56.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:23:56.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-12T18:23:56.005 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-12T18:23:56.322 INFO:tasks.workunit.client.0.trial116.stderr:noout is unset 2026-02-12T18:23:56.333 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-76821-1-0 2026-02-12T18:23:56.333 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-12T18:23:56.333 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-12T18:23:56.333 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-76821-1-0 2026-02-12T18:23:56.333 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-12T18:23:56.334 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-12T18:23:56.334 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-76821-1-0 td/test-erasure-eio/COPY 2026-02-12T18:23:56.351 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-12T18:23:56.352 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-12T18:23:56.353 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 1 + 1 2026-02-12T18:23:56.356 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=2 2026-02-12T18:23:56.356 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-76821-1-0 td/test-erasure-eio 2 0 set 2026-02-12T18:23:56.356 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-76821-1-0 2026-02-12T18:23:56.356 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-12T18:23:56.356 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-12T18:23:56.356 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-12T18:23:56.356 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=2 2026-02-12T18:23:56.356 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-12T18:23:56.356 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-12T18:23:56.357 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-12T18:23:56.357 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-12T18:23:56.357 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-12T18:23:56.357 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-76821-1-0 2026-02-12T18:23:56.357 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:23:56.357 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-76821-1-0 2026-02-12T18:23:56.357 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-76821-1-0 2026-02-12T18:23:56.357 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:23:56.584 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:23:56.585 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:23:56.585 INFO:tasks.workunit.client.0.trial116.stderr:2' 2026-02-12T18:23:56.585 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-12T18:23:56.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-12T18:23:56.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-12T18:23:56.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=2 2026-02-12T18:23:56.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-12T18:23:56.899 INFO:tasks.workunit.client.0.trial116.stderr:noout is set 2026-02-12T18:23:56.909 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-12T18:23:56.909 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-12T18:23:56.909 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-12T18:23:56.911 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-76821-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:23:56.911 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-12T18:23:56.911 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-12T18:23:56.911 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-12T18:23:56.911 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-12T18:23:56.911 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-76821-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:23:56.911 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-12T18:23:56.912 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-12T18:23:56.912 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-12T18:23:56.912 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-12T18:23:56.912 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-12T18:23:56.912 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:23:56.912 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:23:56.912 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:23:56.912 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:23:56.912 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:23:57.218 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:23:57.218 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-76821-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:23:57.218 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-12T18:23:57.218 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-12T18:23:57.218 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-12T18:23:57.219 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-12T18:23:57.219 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-12T18:23:57.219 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-76821-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:23:57.356 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-12T18:23:57.357 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-12T18:23:57.357 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T18:23:57.357 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-12T18:23:57.357 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T18:23:57.357 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-12T18:23:57.357 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:23:57.357 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:23:57.357 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:23:57.357 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:23:57.357 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-12T18:23:57.358 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-12T18:23:57.358 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T18:23:57.358 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T18:23:57.358 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:23:57.358 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T18:23:57.358 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:23:57.358 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:23:57.358 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:23:57.358 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:23:57.358 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:23:57.358 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:23:57.358 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:23:57.359 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:23:57.359 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:23:57.359 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:23:57.359 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:23:57.359 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:23:57.359 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:23:57.359 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:23:57.359 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T18:23:57.359 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-12T18:23:57.359 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-12T18:23:57.359 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-12T18:23:57.359 INFO:tasks.workunit.client.0.trial116.stderr:start osd.2 2026-02-12T18:23:57.360 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:23:57.360 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-12T18:23:57.360 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-12T18:23:57.361 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T18:23:57.361 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T18:23:57.361 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T18:23:57.371 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:57.368+0000 7fc5db462780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:57.372 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:57.369+0000 7fc5db462780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:57.373 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:57.370+0000 7fc5db462780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:23:57.500 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:57.498+0000 7fc5db462780 -1 Falling back to public interface 2026-02-12T18:23:57.584 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-12T18:23:57.584 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:23:57.584 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T18:23:57.584 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:23:57.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:23:57.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:23:57.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:23:57.585 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:23:57.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:23:57.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:23:57.814 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:23:57.893 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:23:57.890+0000 7fc5db462780 -1 osd.2 63 log_to_monitors true 2026-02-12T18:23:58.816 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:23:58.816 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:23:58.816 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:23:58.816 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:23:58.816 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:23:58.816 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:23:59.045 INFO:tasks.workunit.client.0.trial116.stderr:osd.2 up in weight 1 up_from 67 up_thru 55 down_at 64 last_clean_interval [48,63) [v2:127.0.0.1:6818/3698373436,v1:127.0.0.1:6819/3698373436] [v2:127.0.0.1:6820/3698373436,v1:127.0.0.1:6821/3698373436] exists,up 0cd3a3c8-cfaa-4d5e-ad0c-fd4c14cc223c 2026-02-12T18:23:59.045 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:23:59.045 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:23:59.046 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:23:59.046 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-12T18:23:59.046 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:23:59.046 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:23:59.046 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:23:59.046 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:23:59.047 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:23:59.047 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:23:59.047 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:23:59.047 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:23:59.047 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:23:59.131 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:23:59.131 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:23:59.132 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:23:59.132 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:23:59.132 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:23:59.132 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:23:59.361 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:23:59.361 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:23:59.361 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:23:59.361 INFO:tasks.workunit.client.0.trial116.stderr:3' 2026-02-12T18:23:59.361 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:23:59.362 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:59.362 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:23:59.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803794 2026-02-12T18:23:59.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803794 2026-02-12T18:23:59.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803794' 2026-02-12T18:23:59.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:59.426 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:23:59.490 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=249108103174 2026-02-12T18:23:59.490 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 249108103174 2026-02-12T18:23:59.490 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803794 1-249108103174' 2026-02-12T18:23:59.490 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:59.490 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:23:59.554 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=287762808835 2026-02-12T18:23:59.554 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 287762808835 2026-02-12T18:23:59.554 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803794 1-249108103174 2-287762808835' 2026-02-12T18:23:59.554 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:23:59.554 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:23:59.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182418 2026-02-12T18:23:59.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182418 2026-02-12T18:23:59.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803794 1-249108103174 2-287762808835 3-107374182418' 2026-02-12T18:23:59.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:23:59.620 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803794 2026-02-12T18:23:59.620 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:23:59.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:23:59.621 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803794 2026-02-12T18:23:59.621 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:23:59.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803794 2026-02-12T18:23:59.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803794' 2026-02-12T18:23:59.622 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.0 seq 25769803794 2026-02-12T18:23:59.623 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:23:59.849 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803794 -lt 25769803794 2026-02-12T18:23:59.850 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:23:59.850 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-249108103174 2026-02-12T18:23:59.850 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:23:59.851 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:23:59.852 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-249108103174 2026-02-12T18:23:59.852 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:23:59.853 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=249108103174 2026-02-12T18:23:59.853 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 249108103174' 2026-02-12T18:23:59.853 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.1 seq 249108103174 2026-02-12T18:23:59.854 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:24:00.080 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 249108103174 -lt 249108103174 2026-02-12T18:24:00.080 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:24:00.081 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-287762808835 2026-02-12T18:24:00.081 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:24:00.082 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:24:00.082 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-287762808835 2026-02-12T18:24:00.082 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:24:00.083 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=287762808835 2026-02-12T18:24:00.084 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 287762808835' 2026-02-12T18:24:00.084 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.2 seq 287762808835 2026-02-12T18:24:00.084 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:24:00.312 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 287762808835 -lt 287762808835 2026-02-12T18:24:00.312 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:24:00.313 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182418 2026-02-12T18:24:00.313 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:24:00.313 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:24:00.314 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182418 2026-02-12T18:24:00.314 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:24:00.315 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182418 2026-02-12T18:24:00.315 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182418' 2026-02-12T18:24:00.316 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.3 seq 107374182418 2026-02-12T18:24:00.316 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:24:00.547 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182416 -lt 107374182418 2026-02-12T18:24:00.548 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:24:01.549 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:24:01.549 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:24:01.779 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182418 -lt 107374182418 2026-02-12T18:24:01.779 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:24:01.780 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:24:01.780 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:24:02.096 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:24:02.096 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:24:02.096 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:24:02.096 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:24:02.097 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:24:02.097 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:24:02.097 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:24:02.097 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:24:02.319 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:24:02.320 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:24:02.320 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:24:02.320 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:24:02.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:24:02.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:24:02.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:24:02.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-12T18:24:02.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-12T18:24:02.956 INFO:tasks.workunit.client.0.trial116.stderr:noout is unset 2026-02-12T18:24:02.965 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-12T18:24:02.965 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:258: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-76821-1-0 2026-02-12T18:24:02.966 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-12T18:24:02.966 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-12T18:24:02.966 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-76821-1-0 2026-02-12T18:24:02.966 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-12T18:24:02.966 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-12T18:24:02.966 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-76821-1-0 td/test-erasure-eio/COPY 2026-02-12T18:24:02.983 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-12T18:24:02.983 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-12T18:24:02.984 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-12T18:24:02.987 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:345: TEST_rados_get_bad_size_shard_1: rados_get_data_bad_size td/test-erasure-eio 1 256 add 2026-02-12T18:24:02.987 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-12T18:24:02.987 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-12T18:24:02.987 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=1 2026-02-12T18:24:02.987 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-12T18:24:02.987 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=256 2026-02-12T18:24:02.987 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-12T18:24:02.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=add 2026-02-12T18:24:02.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-12T18:24:02.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-76821-1-256 2026-02-12T18:24:02.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-76821-1-256 2026-02-12T18:24:02.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-12T18:24:02.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-12T18:24:02.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-76821-1-256 2026-02-12T18:24:02.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:24:02.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-12T18:24:02.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:24:02.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-12T18:24:02.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:24:02.989 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-12T18:24:02.989 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:24:02.989 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-12T18:24:02.989 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-76821-1-256 td/test-erasure-eio/ORIGINAL 2026-02-12T18:24:03.003 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-76821-1-256 td/test-erasure-eio 1 256 add 2026-02-12T18:24:03.003 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-76821-1-256 2026-02-12T18:24:03.003 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-12T18:24:03.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-12T18:24:03.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-12T18:24:03.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-12T18:24:03.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-12T18:24:03.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-12T18:24:03.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-12T18:24:03.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-12T18:24:03.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-12T18:24:03.004 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-76821-1-256 2026-02-12T18:24:03.004 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:24:03.004 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-76821-1-256 2026-02-12T18:24:03.004 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-76821-1-256 2026-02-12T18:24:03.005 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:24:03.233 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:24:03.233 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:24:03.234 INFO:tasks.workunit.client.0.trial116.stderr:2' 2026-02-12T18:24:03.234 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-12T18:24:03.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-12T18:24:03.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-12T18:24:03.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-12T18:24:03.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-12T18:24:03.514 INFO:tasks.workunit.client.0.trial116.stderr:noout is set 2026-02-12T18:24:03.524 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-12T18:24:03.525 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-76821-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:24:03.525 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-12T18:24:03.525 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-12T18:24:03.525 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-12T18:24:03.525 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-12T18:24:03.525 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-76821-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:24:03.525 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-12T18:24:03.525 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-12T18:24:03.525 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-12T18:24:03.526 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-12T18:24:03.526 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-12T18:24:03.526 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:24:03.526 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:24:03.526 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:24:03.526 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:24:03.526 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:24:03.831 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:24:03.832 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-76821-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:24:03.832 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-12T18:24:03.832 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-12T18:24:03.832 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-12T18:24:03.832 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-12T18:24:03.832 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-12T18:24:03.832 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-76821-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:24:03.920 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-12T18:24:03.920 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-12T18:24:03.920 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T18:24:03.920 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-12T18:24:03.920 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T18:24:03.920 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-12T18:24:03.921 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:24:03.921 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:24:03.921 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:24:03.921 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:24:03.921 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-12T18:24:03.921 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-12T18:24:03.921 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T18:24:03.921 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T18:24:03.921 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:24:03.921 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T18:24:03.921 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:24:03.921 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:24:03.922 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:24:03.922 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:03.922 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:03.922 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:24:03.922 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:24:03.922 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:24:03.922 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:24:03.922 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:24:03.922 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:24:03.922 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:24:03.922 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:24:03.922 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:24:03.923 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T18:24:03.923 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-12T18:24:03.923 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-12T18:24:03.923 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-12T18:24:03.923 INFO:tasks.workunit.client.0.trial116.stderr:start osd.1 2026-02-12T18:24:03.923 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:24:03.923 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-12T18:24:03.924 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-12T18:24:03.924 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T18:24:03.924 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T18:24:03.924 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T18:24:03.934 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:03.930+0000 7f57d9c08780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:03.935 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:03.932+0000 7f57d9c08780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:03.936 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:03.933+0000 7f57d9c08780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:04.068 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:04.065+0000 7f57d9c08780 -1 Falling back to public interface 2026-02-12T18:24:04.148 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-12T18:24:04.149 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:24:04.149 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T18:24:04.149 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:24:04.149 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:24:04.149 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:24:04.149 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:24:04.149 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:24:04.149 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:24:04.149 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:24:04.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:24:04.392 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:04.389+0000 7f57d9c08780 -1 osd.1 71 log_to_monitors true 2026-02-12T18:24:05.380 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:24:05.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:24:05.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:24:05.381 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:24:05.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:24:05.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:24:05.605 INFO:tasks.workunit.client.0.trial116.stderr:osd.1 up in weight 1 up_from 76 up_thru 76 down_at 73 last_clean_interval [58,71) [v2:127.0.0.1:6810/2839492087,v1:127.0.0.1:6811/2839492087] [v2:127.0.0.1:6812/2839492087,v1:127.0.0.1:6813/2839492087] exists,up bf1f6743-0a0a-4477-a288-553d68123a01 2026-02-12T18:24:05.605 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:24:05.605 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:24:05.605 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:24:05.606 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-12T18:24:05.606 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:24:05.606 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:24:05.606 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:24:05.606 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:24:05.607 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:24:05.607 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:24:05.607 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:24:05.607 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:24:05.607 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:24:05.692 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:24:05.693 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:24:05.693 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:24:05.693 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:24:05.693 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:24:05.693 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:24:05.916 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:24:05.916 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:24:05.916 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:24:05.916 INFO:tasks.workunit.client.0.trial116.stderr:3' 2026-02-12T18:24:05.916 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:24:05.916 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:24:05.917 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:24:05.983 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803797 2026-02-12T18:24:05.983 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803797 2026-02-12T18:24:05.983 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803797' 2026-02-12T18:24:05.983 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:24:05.983 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:24:06.048 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=326417514499 2026-02-12T18:24:06.049 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 326417514499 2026-02-12T18:24:06.049 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803797 1-326417514499' 2026-02-12T18:24:06.049 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:24:06.049 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:24:06.112 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=287762808838 2026-02-12T18:24:06.112 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 287762808838 2026-02-12T18:24:06.112 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803797 1-326417514499 2-287762808838' 2026-02-12T18:24:06.113 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:24:06.113 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:24:06.177 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182421 2026-02-12T18:24:06.177 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182421 2026-02-12T18:24:06.177 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803797 1-326417514499 2-287762808838 3-107374182421' 2026-02-12T18:24:06.177 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:24:06.177 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803797 2026-02-12T18:24:06.178 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:24:06.179 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:24:06.180 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803797 2026-02-12T18:24:06.180 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:24:06.181 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803797 2026-02-12T18:24:06.181 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803797' 2026-02-12T18:24:06.181 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.0 seq 25769803797 2026-02-12T18:24:06.181 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:24:06.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803795 -lt 25769803797 2026-02-12T18:24:06.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:24:07.411 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:24:07.411 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:24:07.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803798 -lt 25769803797 2026-02-12T18:24:07.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:24:07.651 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-326417514499 2026-02-12T18:24:07.652 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:24:07.652 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:24:07.653 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-326417514499 2026-02-12T18:24:07.653 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:24:07.654 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=326417514499 2026-02-12T18:24:07.654 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 326417514499' 2026-02-12T18:24:07.654 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.1 seq 326417514499 2026-02-12T18:24:07.655 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:24:07.882 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 326417514499 -lt 326417514499 2026-02-12T18:24:07.882 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:24:07.883 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-287762808838 2026-02-12T18:24:07.883 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:24:07.883 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:24:07.884 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-287762808838 2026-02-12T18:24:07.884 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:24:07.885 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=287762808838 2026-02-12T18:24:07.886 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 287762808838' 2026-02-12T18:24:07.886 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.2 seq 287762808838 2026-02-12T18:24:07.886 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:24:08.112 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 287762808838 -lt 287762808838 2026-02-12T18:24:08.112 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:24:08.113 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182421 2026-02-12T18:24:08.113 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:24:08.113 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:24:08.114 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182421 2026-02-12T18:24:08.114 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:24:08.115 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182421 2026-02-12T18:24:08.115 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182421' 2026-02-12T18:24:08.115 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.3 seq 107374182421 2026-02-12T18:24:08.116 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:24:08.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182421 -lt 107374182421 2026-02-12T18:24:08.347 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:24:08.347 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:24:08.347 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:24:08.665 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:24:08.665 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:24:08.666 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:24:08.667 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:24:08.667 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:24:08.667 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:24:08.667 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:24:08.667 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:24:08.887 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:24:08.887 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:24:08.887 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:24:08.887 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:24:09.205 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:24:09.205 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:24:09.205 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:24:09.206 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-12T18:24:09.206 INFO:tasks.workunit.client.0.trial116.stderr:1+0 records in 2026-02-12T18:24:09.206 INFO:tasks.workunit.client.0.trial116.stderr:1+0 records out 2026-02-12T18:24:09.206 INFO:tasks.workunit.client.0.trial116.stderr:256 bytes copied, 1.9656e-05 s, 13.0 MB/s 2026-02-12T18:24:09.206 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-76821-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:24:09.206 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-12T18:24:09.206 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-12T18:24:09.206 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-12T18:24:09.206 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-12T18:24:09.206 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-76821-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:24:09.206 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-12T18:24:09.206 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-12T18:24:09.206 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-12T18:24:09.206 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-12T18:24:09.206 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-12T18:24:09.206 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:24:09.206 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:24:09.206 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:24:09.206 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:24:09.206 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:24:09.310 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:24:09.310 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-76821-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:24:09.310 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-12T18:24:09.310 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-12T18:24:09.310 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-12T18:24:09.310 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-12T18:24:09.310 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-12T18:24:09.310 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-76821-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:24:09.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-12T18:24:09.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-12T18:24:09.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T18:24:09.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-12T18:24:09.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T18:24:09.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-12T18:24:09.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:24:09.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:24:09.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:24:09.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:24:09.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-12T18:24:09.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-12T18:24:09.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T18:24:09.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T18:24:09.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:24:09.443 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T18:24:09.444 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:24:09.444 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:24:09.444 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:24:09.444 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:09.444 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:09.444 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:24:09.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:24:09.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:24:09.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:24:09.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:24:09.444 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:24:09.445 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:24:09.445 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:24:09.445 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:24:09.445 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T18:24:09.445 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-12T18:24:09.445 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-12T18:24:09.445 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-12T18:24:09.445 INFO:tasks.workunit.client.0.trial116.stderr:start osd.1 2026-02-12T18:24:09.445 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:24:09.445 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-12T18:24:09.445 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-12T18:24:09.446 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T18:24:09.446 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T18:24:09.446 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T18:24:09.456 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:09.452+0000 7fa2ce22f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:09.457 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:09.453+0000 7fa2ce22f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:09.458 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:09.454+0000 7fa2ce22f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:09.606 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:09.603+0000 7fa2ce22f780 -1 Falling back to public interface 2026-02-12T18:24:09.670 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-12T18:24:09.671 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:24:09.671 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T18:24:09.671 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:24:09.671 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:24:09.671 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:24:09.671 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:24:09.671 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:24:09.671 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:24:09.671 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:24:09.901 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:24:09.916 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:09.912+0000 7fa2ce22f780 -1 osd.1 77 log_to_monitors true 2026-02-12T18:24:10.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:24:10.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:24:10.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:24:10.903 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:24:10.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:24:10.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:24:11.130 INFO:tasks.workunit.client.0.trial116.stderr:osd.1 up in weight 1 up_from 81 up_thru 81 down_at 78 last_clean_interval [76,77) [v2:127.0.0.1:6810/3121177992,v1:127.0.0.1:6811/3121177992] [v2:127.0.0.1:6812/3121177992,v1:127.0.0.1:6813/3121177992] exists,up bf1f6743-0a0a-4477-a288-553d68123a01 2026-02-12T18:24:11.130 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:24:11.131 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:24:11.131 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:24:11.131 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-12T18:24:11.131 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:24:11.131 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:24:11.131 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:24:11.131 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:24:11.131 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:24:11.132 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:24:11.132 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:24:11.132 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:24:11.133 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:24:11.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:24:11.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:24:11.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:24:11.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:24:11.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:24:11.218 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:24:11.446 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:24:11.446 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:24:11.446 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:24:11.446 INFO:tasks.workunit.client.0.trial116.stderr:3' 2026-02-12T18:24:11.446 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:24:11.446 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:24:11.447 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:24:11.512 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803801 2026-02-12T18:24:11.512 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803801 2026-02-12T18:24:11.512 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803801' 2026-02-12T18:24:11.512 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:24:11.512 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:24:11.575 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=347892350979 2026-02-12T18:24:11.575 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 347892350979 2026-02-12T18:24:11.575 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803801 1-347892350979' 2026-02-12T18:24:11.575 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:24:11.575 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:24:11.645 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=287762808841 2026-02-12T18:24:11.645 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 287762808841 2026-02-12T18:24:11.645 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803801 1-347892350979 2-287762808841' 2026-02-12T18:24:11.645 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:24:11.645 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:24:11.709 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182424 2026-02-12T18:24:11.709 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182424 2026-02-12T18:24:11.709 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803801 1-347892350979 2-287762808841 3-107374182424' 2026-02-12T18:24:11.710 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:24:11.710 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803801 2026-02-12T18:24:11.710 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:24:11.711 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:24:11.711 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803801 2026-02-12T18:24:11.711 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:24:11.713 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803801 2026-02-12T18:24:11.713 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803801' 2026-02-12T18:24:11.713 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.0 seq 25769803801 2026-02-12T18:24:11.713 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:24:11.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803801 -lt 25769803801 2026-02-12T18:24:11.940 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:24:11.940 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-347892350979 2026-02-12T18:24:11.941 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:24:11.941 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:24:11.942 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-347892350979 2026-02-12T18:24:11.942 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:24:11.943 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=347892350979 2026-02-12T18:24:11.943 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 347892350979' 2026-02-12T18:24:11.943 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.1 seq 347892350979 2026-02-12T18:24:11.943 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:24:12.168 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 1 -lt 347892350979 2026-02-12T18:24:12.169 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:24:13.169 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:24:13.170 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:24:13.398 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 1 -lt 347892350979 2026-02-12T18:24:13.399 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:24:14.399 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-12T18:24:14.400 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:24:14.634 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 347892350979 -lt 347892350979 2026-02-12T18:24:14.634 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:24:14.634 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-287762808841 2026-02-12T18:24:14.635 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:24:14.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:24:14.636 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-287762808841 2026-02-12T18:24:14.636 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:24:14.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=287762808841 2026-02-12T18:24:14.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 287762808841' 2026-02-12T18:24:14.638 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.2 seq 287762808841 2026-02-12T18:24:14.638 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:24:14.863 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 287762808842 -lt 287762808841 2026-02-12T18:24:14.863 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:24:14.864 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182424 2026-02-12T18:24:14.864 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:24:14.865 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:24:14.865 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182424 2026-02-12T18:24:14.865 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:24:14.866 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182424 2026-02-12T18:24:14.866 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182424' 2026-02-12T18:24:14.867 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.3 seq 107374182424 2026-02-12T18:24:14.867 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:24:15.091 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182424 -lt 107374182424 2026-02-12T18:24:15.092 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:24:15.092 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:24:15.092 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:24:15.415 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:24:15.415 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:24:15.415 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:24:15.415 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:24:15.415 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:24:15.415 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:24:15.415 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:24:15.415 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:24:15.638 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:24:15.639 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:24:15.639 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:24:15.640 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:24:15.963 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:24:15.964 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:24:15.964 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:24:15.964 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-12T18:24:15.964 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-12T18:24:16.281 INFO:tasks.workunit.client.0.trial116.stderr:noout is unset 2026-02-12T18:24:16.293 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-76821-1-256 2026-02-12T18:24:16.294 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-12T18:24:16.294 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-12T18:24:16.294 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-76821-1-256 2026-02-12T18:24:16.294 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-12T18:24:16.294 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-12T18:24:16.294 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-76821-1-256 td/test-erasure-eio/COPY 2026-02-12T18:24:16.311 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-12T18:24:16.312 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-12T18:24:16.313 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 1 + 1 2026-02-12T18:24:16.315 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=2 2026-02-12T18:24:16.316 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-76821-1-256 td/test-erasure-eio 2 256 add 2026-02-12T18:24:16.316 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-76821-1-256 2026-02-12T18:24:16.316 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-12T18:24:16.316 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-12T18:24:16.316 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-12T18:24:16.316 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=2 2026-02-12T18:24:16.316 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-12T18:24:16.316 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-12T18:24:16.316 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-12T18:24:16.316 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-12T18:24:16.317 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-12T18:24:16.317 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-76821-1-256 2026-02-12T18:24:16.317 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:24:16.317 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-76821-1-256 2026-02-12T18:24:16.317 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-76821-1-256 2026-02-12T18:24:16.317 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:24:16.545 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:24:16.546 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:24:16.546 INFO:tasks.workunit.client.0.trial116.stderr:2' 2026-02-12T18:24:16.546 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-12T18:24:16.546 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-12T18:24:16.546 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-12T18:24:16.546 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=2 2026-02-12T18:24:16.546 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-12T18:24:16.861 INFO:tasks.workunit.client.0.trial116.stderr:noout is set 2026-02-12T18:24:16.874 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-12T18:24:16.874 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-76821-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:24:16.874 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-12T18:24:16.874 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-12T18:24:16.874 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-12T18:24:16.875 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-12T18:24:16.875 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-76821-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:24:16.875 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-12T18:24:16.875 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-12T18:24:16.875 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-12T18:24:16.875 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-12T18:24:16.875 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-12T18:24:16.875 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:24:16.875 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:24:16.875 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:24:16.875 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:24:16.876 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:24:17.181 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:24:17.181 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-76821-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:24:17.181 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-12T18:24:17.181 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-12T18:24:17.182 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-12T18:24:17.182 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-12T18:24:17.182 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-12T18:24:17.182 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-76821-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:24:17.283 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-12T18:24:17.283 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-12T18:24:17.283 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T18:24:17.283 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-12T18:24:17.283 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T18:24:17.283 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-12T18:24:17.283 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:24:17.284 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:24:17.284 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:24:17.284 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:24:17.284 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-12T18:24:17.284 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-12T18:24:17.284 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T18:24:17.284 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T18:24:17.284 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:24:17.284 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T18:24:17.284 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:24:17.284 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:24:17.285 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:24:17.285 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:17.285 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:17.285 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:24:17.285 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:24:17.285 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:24:17.285 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:24:17.285 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:24:17.285 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:24:17.285 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:24:17.285 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:24:17.285 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:24:17.285 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T18:24:17.286 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-12T18:24:17.286 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-12T18:24:17.286 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-12T18:24:17.286 INFO:tasks.workunit.client.0.trial116.stderr:start osd.2 2026-02-12T18:24:17.286 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:24:17.286 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-12T18:24:17.287 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-12T18:24:17.287 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T18:24:17.287 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T18:24:17.287 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T18:24:17.297 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:17.293+0000 7fa1a1831780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:17.298 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:17.295+0000 7fa1a1831780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:17.299 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:17.296+0000 7fa1a1831780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:17.436 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:17.433+0000 7fa1a1831780 -1 Falling back to public interface 2026-02-12T18:24:17.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-12T18:24:17.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:24:17.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T18:24:17.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:24:17.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:24:17.508 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:24:17.509 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:24:17.509 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:24:17.509 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:24:17.509 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:24:17.732 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:24:17.766 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:17.763+0000 7fa1a1831780 -1 osd.2 85 log_to_monitors true 2026-02-12T18:24:18.734 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:24:18.734 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:24:18.734 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:24:18.734 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:24:18.734 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:24:18.735 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:24:18.960 INFO:tasks.workunit.client.0.trial116.stderr:osd.2 up in weight 1 up_from 90 up_thru 78 down_at 87 last_clean_interval [67,85) [v2:127.0.0.1:6818/3926148694,v1:127.0.0.1:6819/3926148694] [v2:127.0.0.1:6820/3926148694,v1:127.0.0.1:6821/3926148694] exists,up 0cd3a3c8-cfaa-4d5e-ad0c-fd4c14cc223c 2026-02-12T18:24:18.960 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:24:18.960 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:24:18.960 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:24:18.960 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-12T18:24:18.960 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:24:18.960 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:24:18.961 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:24:18.961 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:24:18.961 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:24:18.961 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:24:18.961 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:24:18.961 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:24:18.961 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:24:19.047 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:24:19.047 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:24:19.047 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:24:19.047 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:24:19.047 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:24:19.048 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:24:19.273 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:24:19.273 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:24:19.273 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:24:19.273 INFO:tasks.workunit.client.0.trial116.stderr:3' 2026-02-12T18:24:19.273 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:24:19.273 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:24:19.273 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:24:19.335 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803804 2026-02-12T18:24:19.335 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803804 2026-02-12T18:24:19.336 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803804' 2026-02-12T18:24:19.336 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:24:19.336 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:24:19.402 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=347892350982 2026-02-12T18:24:19.402 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 347892350982 2026-02-12T18:24:19.402 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803804 1-347892350982' 2026-02-12T18:24:19.402 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:24:19.402 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:24:19.467 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=386547056643 2026-02-12T18:24:19.468 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 386547056643 2026-02-12T18:24:19.468 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803804 1-347892350982 2-386547056643' 2026-02-12T18:24:19.468 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:24:19.468 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:24:19.532 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182428 2026-02-12T18:24:19.533 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182428 2026-02-12T18:24:19.533 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803804 1-347892350982 2-386547056643 3-107374182428' 2026-02-12T18:24:19.533 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:24:19.534 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803804 2026-02-12T18:24:19.534 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:24:19.534 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:24:19.535 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803804 2026-02-12T18:24:19.535 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:24:19.536 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803804 2026-02-12T18:24:19.536 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803804' 2026-02-12T18:24:19.536 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.0 seq 25769803804 2026-02-12T18:24:19.536 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:24:19.761 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803804 -lt 25769803804 2026-02-12T18:24:19.762 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:24:19.762 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-347892350982 2026-02-12T18:24:19.762 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:24:19.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:24:19.764 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-347892350982 2026-02-12T18:24:19.764 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:24:19.765 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=347892350982 2026-02-12T18:24:19.765 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 347892350982' 2026-02-12T18:24:19.765 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.1 seq 347892350982 2026-02-12T18:24:19.765 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:24:19.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 347892350982 -lt 347892350982 2026-02-12T18:24:19.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:24:19.994 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-386547056643 2026-02-12T18:24:19.994 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:24:19.994 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:24:19.995 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-386547056643 2026-02-12T18:24:19.996 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:24:19.996 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=386547056643 2026-02-12T18:24:19.997 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 386547056643' 2026-02-12T18:24:19.997 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.2 seq 386547056643 2026-02-12T18:24:19.997 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:24:20.222 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 386547056643 -lt 386547056643 2026-02-12T18:24:20.222 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:24:20.223 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182428 2026-02-12T18:24:20.223 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:24:20.224 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:24:20.225 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182428 2026-02-12T18:24:20.225 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:24:20.226 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182428 2026-02-12T18:24:20.226 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182428' 2026-02-12T18:24:20.226 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.3 seq 107374182428 2026-02-12T18:24:20.226 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:24:20.453 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182428 -lt 107374182428 2026-02-12T18:24:20.454 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:24:20.454 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:24:20.455 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:24:20.785 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:24:20.785 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:24:20.785 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:24:20.785 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:24:20.785 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:24:20.786 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:24:20.786 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:24:20.786 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:24:21.011 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:24:21.012 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:24:21.013 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:24:21.013 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:24:21.342 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:24:21.343 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:24:21.343 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:24:21.343 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-12T18:24:21.345 INFO:tasks.workunit.client.0.trial116.stderr:1+0 records in 2026-02-12T18:24:21.345 INFO:tasks.workunit.client.0.trial116.stderr:1+0 records out 2026-02-12T18:24:21.345 INFO:tasks.workunit.client.0.trial116.stderr:256 bytes copied, 2.0768e-05 s, 12.3 MB/s 2026-02-12T18:24:21.345 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-76821-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:24:21.345 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-12T18:24:21.345 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-12T18:24:21.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-12T18:24:21.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-12T18:24:21.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-76821-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:24:21.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-12T18:24:21.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-12T18:24:21.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-12T18:24:21.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-12T18:24:21.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-12T18:24:21.346 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:24:21.346 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:24:21.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:24:21.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:24:21.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:24:21.450 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:24:21.450 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-76821-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:24:21.450 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-12T18:24:21.451 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-12T18:24:21.451 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-12T18:24:21.451 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-12T18:24:21.451 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-12T18:24:21.451 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-76821-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-12T18:24:21.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-12T18:24:21.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-12T18:24:21.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T18:24:21.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-12T18:24:21.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T18:24:21.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-12T18:24:21.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:24:21.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:24:21.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:24:21.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:24:21.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-12T18:24:21.588 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-12T18:24:21.588 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T18:24:21.588 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T18:24:21.588 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:24:21.588 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T18:24:21.588 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:24:21.588 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:24:21.588 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:24:21.588 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:21.588 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:21.588 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:24:21.588 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:24:21.588 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:24:21.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:24:21.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:24:21.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:24:21.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:24:21.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:24:21.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:24:21.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T18:24:21.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-12T18:24:21.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-12T18:24:21.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-12T18:24:21.589 INFO:tasks.workunit.client.0.trial116.stderr:start osd.2 2026-02-12T18:24:21.590 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:24:21.590 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-12T18:24:21.591 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-12T18:24:21.591 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T18:24:21.591 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T18:24:21.591 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T18:24:21.602 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:21.598+0000 7f4b0f020780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:21.603 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:21.599+0000 7f4b0f020780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:21.604 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:21.600+0000 7f4b0f020780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:21.734 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:21.730+0000 7f4b0f020780 -1 Falling back to public interface 2026-02-12T18:24:21.816 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-12T18:24:21.816 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:24:21.816 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T18:24:21.816 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:24:21.816 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:24:21.816 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:24:21.816 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:24:21.816 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:24:21.816 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:24:21.817 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:24:22.041 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:24:22.074 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:22.070+0000 7f4b0f020780 -1 osd.2 91 log_to_monitors true 2026-02-12T18:24:23.042 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:24:23.042 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:24:23.042 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:24:23.042 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:24:23.043 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:24:23.043 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:24:23.270 INFO:tasks.workunit.client.0.trial116.stderr:osd.2 up in weight 1 up_from 95 up_thru 78 down_at 92 last_clean_interval [90,91) [v2:127.0.0.1:6818/2416611637,v1:127.0.0.1:6819/2416611637] [v2:127.0.0.1:6820/2416611637,v1:127.0.0.1:6821/2416611637] exists,up 0cd3a3c8-cfaa-4d5e-ad0c-fd4c14cc223c 2026-02-12T18:24:23.270 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:24:23.270 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:24:23.270 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:24:23.271 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-12T18:24:23.271 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:24:23.271 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:24:23.271 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:24:23.271 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:24:23.272 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:24:23.272 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:24:23.272 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:24:23.272 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:24:23.272 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:24:23.355 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:24:23.355 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:24:23.355 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:24:23.355 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:24:23.355 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:24:23.355 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:24:23.581 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:24:23.581 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:24:23.581 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:24:23.581 INFO:tasks.workunit.client.0.trial116.stderr:3' 2026-02-12T18:24:23.581 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:24:23.581 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:24:23.581 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:24:23.647 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803807 2026-02-12T18:24:23.647 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803807 2026-02-12T18:24:23.647 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803807' 2026-02-12T18:24:23.647 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:24:23.647 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:24:23.713 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=347892350985 2026-02-12T18:24:23.713 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 347892350985 2026-02-12T18:24:23.713 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803807 1-347892350985' 2026-02-12T18:24:23.713 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:24:23.714 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:24:23.780 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=408021893123 2026-02-12T18:24:23.780 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 408021893123 2026-02-12T18:24:23.780 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803807 1-347892350985 2-408021893123' 2026-02-12T18:24:23.780 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:24:23.780 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:24:23.844 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182431 2026-02-12T18:24:23.844 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182431 2026-02-12T18:24:23.845 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803807 1-347892350985 2-408021893123 3-107374182431' 2026-02-12T18:24:23.845 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:24:23.845 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803807 2026-02-12T18:24:23.845 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:24:23.846 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:24:23.846 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803807 2026-02-12T18:24:23.846 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:24:23.847 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803807 2026-02-12T18:24:23.847 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803807' 2026-02-12T18:24:23.847 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.0 seq 25769803807 2026-02-12T18:24:23.848 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:24:24.071 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803805 -lt 25769803807 2026-02-12T18:24:24.072 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:24:25.072 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:24:25.073 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:24:25.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803805 -lt 25769803807 2026-02-12T18:24:25.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:24:26.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-12T18:24:26.300 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:24:26.527 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803807 -lt 25769803807 2026-02-12T18:24:26.527 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:24:26.528 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-347892350985 2026-02-12T18:24:26.528 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:24:26.529 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:24:26.530 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-347892350985 2026-02-12T18:24:26.530 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:24:26.531 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=347892350985 2026-02-12T18:24:26.531 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 347892350985' 2026-02-12T18:24:26.531 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.1 seq 347892350985 2026-02-12T18:24:26.531 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:24:26.758 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 347892350986 -lt 347892350985 2026-02-12T18:24:26.758 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:24:26.759 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-408021893123 2026-02-12T18:24:26.759 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:24:26.760 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:24:26.760 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-408021893123 2026-02-12T18:24:26.761 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:24:26.762 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=408021893123 2026-02-12T18:24:26.762 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 408021893123' 2026-02-12T18:24:26.762 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.2 seq 408021893123 2026-02-12T18:24:26.762 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:24:26.987 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 408021893123 -lt 408021893123 2026-02-12T18:24:26.987 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:24:26.988 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-107374182431 2026-02-12T18:24:26.988 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:24:26.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:24:26.989 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182431 2026-02-12T18:24:26.989 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:24:26.990 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=107374182431 2026-02-12T18:24:26.990 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 107374182431' 2026-02-12T18:24:26.990 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.3 seq 107374182431 2026-02-12T18:24:26.991 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:24:27.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 107374182431 -lt 107374182431 2026-02-12T18:24:27.216 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:24:27.216 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:24:27.216 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:24:27.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:24:27.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:24:27.585 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:24:27.585 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:24:27.585 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:24:27.585 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:24:27.585 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:24:27.585 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:24:27.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:24:27.810 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:24:27.810 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:24:27.810 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:24:28.125 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:24:28.126 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:24:28.126 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:24:28.126 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-12T18:24:28.126 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-12T18:24:28.445 INFO:tasks.workunit.client.0.trial116.stderr:noout is unset 2026-02-12T18:24:28.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-12T18:24:28.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:258: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-76821-1-256 2026-02-12T18:24:28.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-12T18:24:28.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-12T18:24:28.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-76821-1-256 2026-02-12T18:24:28.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-12T18:24:28.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-12T18:24:28.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-76821-1-256 td/test-erasure-eio/COPY 2026-02-12T18:24:28.473 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-12T18:24:28.473 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-12T18:24:28.474 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-12T18:24:28.475 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:346: TEST_rados_get_bad_size_shard_1: delete_erasure_coded_pool pool-jerasure 2026-02-12T18:24:28.475 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-12T18:24:28.476 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-12T18:24:28.716 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-jerasure' does not exist 2026-02-12T18:24:28.724 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-12T18:24:29.003 INFO:tasks.workunit.client.0.trial116.stderr:erasure-code-profile myprofile does not exist 2026-02-12T18:24:29.011 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-12T18:24:29.011 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-12T18:24:29.011 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:24:29.011 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-12T18:24:29.011 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:24:29.011 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:24:29.012 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:24:29.012 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:24:29.012 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:24:29.121 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:24:29.122 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:24:29.123 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:24:29.123 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:24:29.124 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:24:29.124 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:24:29.125 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:24:29.125 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:24:29.126 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:24:29.126 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:24:29.126 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:24:29.127 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:24:29.128 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:24:29.128 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-12T18:24:29.145 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:24:29.145 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:29.145 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:29.145 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76821 2026-02-12T18:24:29.146 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:24:29.147 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:24:29.147 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-12T18:24:29.147 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-12T18:24:29.147 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-12T18:24:29.147 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-12T18:24:29.147 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-12T18:24:29.147 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:24:29.147 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-12T18:24:29.147 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:24:29.147 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:24:29.148 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:24:29.148 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:24:29.148 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:24:29.148 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:24:29.148 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:24:29.149 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:24:29.149 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:24:29.150 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:24:29.151 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:24:29.151 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:24:29.152 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:24:29.152 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:24:29.152 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:24:29.152 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:24:29.153 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:24:29.154 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:24:29.154 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-12T18:24:29.155 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:24:29.155 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:29.156 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:29.156 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76821 2026-02-12T18:24:29.156 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:24:29.156 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:24:29.156 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-12T18:24:29.158 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T18:24:29.158 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:29.158 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:29.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76821 2026-02-12T18:24:29.159 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T18:24:29.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T18:24:29.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T18:24:29.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-12T18:24:29.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-12T18:24:29.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-12T18:24:29.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T18:24:29.161 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T18:24:29.161 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T18:24:29.161 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-12T18:24:29.161 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-12T18:24:29.178 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T18:24:29.178 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:24:29.178 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:24:29.178 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:24:29.178 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:29.178 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:29.178 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:24:29.179 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T18:24:29.201 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T18:24:29.202 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T18:24:29.203 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:24:29.203 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:24:29.203 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T18:24:29.204 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T18:24:29.204 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:24:29.204 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:24:29.204 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:24:29.204 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:24:29.204 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:29.205 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:29.205 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:24:29.205 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:24:29.205 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76821/ceph-mon.a.asok config get fsid 2026-02-12T18:24:29.247 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T18:24:29.247 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:24:29.247 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:24:29.247 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T18:24:29.247 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T18:24:29.248 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:24:29.248 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:24:29.248 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:24:29.249 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:24:29.249 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:29.249 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:29.249 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:24:29.249 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:24:29.249 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76821/ceph-mon.a.asok config get mon_host 2026-02-12T18:24:29.290 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-12T18:24:29.290 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-12T18:24:29.290 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T18:24:29.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T18:24:29.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T18:24:29.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-12T18:24:29.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T18:24:29.389 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T18:24:29.389 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:24:29.389 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:24:29.390 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:24:29.390 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:29.390 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:29.390 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:24:29.390 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:24:29.391 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:24:29.407 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-12T18:24:29.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T18:24:29.505 INFO:tasks.workunit.client.0.trial116.stderr:pool 'rbd' already exists 2026-02-12T18:24:29.512 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:24:30.515 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-12T18:24:30.515 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:24:30.515 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:24:30.516 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:24:30.516 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:30.516 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:30.516 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:24:30.516 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-12T18:24:30.516 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-mon.a.asok log flush 2026-02-12T18:24:30.554 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:24:30.560 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-12T18:24:30.561 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:24:29.191+0000 7efda2506c00 0 load: jerasure load: lrc 2026-02-12T18:24:30.562 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_subread_eio_shard_0 td/test-erasure-eio 2026-02-12T18:24:30.562 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:275: TEST_rados_get_subread_eio_shard_0: local dir=td/test-erasure-eio 2026-02-12T18:24:30.562 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:276: TEST_rados_get_subread_eio_shard_0: setup_osds 4 2026-02-12T18:24:30.562 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-12T18:24:30.562 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-12T18:24:30.562 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-12T18:24:30.563 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-12T18:24:30.567 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:24:30.567 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-12T18:24:30.567 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:24:30.567 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:24:30.567 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T18:24:30.567 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:24:30.568 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-12T18:24:30.568 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:24:30.568 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:24:30.568 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:24:30.568 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:24:30.568 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-12T18:24:30.568 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-12T18:24:30.568 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:24:30.568 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:24:30.568 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:24:30.568 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:24:30.568 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:24:30.569 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:24:30.569 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:24:30.569 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:30.569 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:30.569 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:24:30.569 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:24:30.569 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:24:30.569 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:24:30.569 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:24:30.569 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:24:30.569 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:24:30.569 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:24:30.569 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:24:30.570 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:24:30.570 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:24:30.570 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:24:30.570 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:24:30.570 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-12T18:24:30.570 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:24:30.570 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=dcd9d85f-3843-42ae-8b0e-fa874509ec83 2026-02-12T18:24:30.570 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 dcd9d85f-3843-42ae-8b0e-fa874509ec83' 2026-02-12T18:24:30.570 INFO:tasks.workunit.client.0.trial116.stdout:add osd0 dcd9d85f-3843-42ae-8b0e-fa874509ec83 2026-02-12T18:24:30.571 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:24:30.578 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDeGo5pEoM3IhAACAZkCpYoiyU7GPGSGLHRoA== 2026-02-12T18:24:30.578 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDeGo5pEoM3IhAACAZkCpYoiyU7GPGSGLHRoA=="}' 2026-02-12T18:24:30.578 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new dcd9d85f-3843-42ae-8b0e-fa874509ec83 -i td/test-erasure-eio/0/new.json 2026-02-12T18:24:30.679 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:24:30.686 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-12T18:24:30.687 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDeGo5pEoM3IhAACAZkCpYoiyU7GPGSGLHRoA== --osd-uuid dcd9d85f-3843-42ae-8b0e-fa874509ec83 2026-02-12T18:24:30.703 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:30.699+0000 7f00cf067780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:30.705 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:30.701+0000 7f00cf067780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:30.706 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:30.702+0000 7f00cf067780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:30.706 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:30.702+0000 7f00cf067780 -1 bdev(0x4270c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-12T18:24:30.706 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:30.702+0000 7f00cf067780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-12T18:24:31.144 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-12T18:24:31.144 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:24:31.145 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T18:24:31.145 INFO:tasks.workunit.client.0.trial116.stdout:adding osd0 key to auth repository 2026-02-12T18:24:31.145 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:24:31.248 INFO:tasks.workunit.client.0.trial116.stdout:start osd.0 2026-02-12T18:24:31.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T18:24:31.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:24:31.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:24:31.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:24:31.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:24:31.263 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:31.259+0000 7f4299866780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:31.265 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:31.262+0000 7f4299866780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:31.266 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:31.263+0000 7f4299866780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:31.347 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T18:24:31.347 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:24:31.347 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T18:24:31.347 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:24:31.347 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:24:31.347 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:24:31.348 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:24:31.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:24:31.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:24:31.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:24:31.445 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:24:31.471 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:31.467+0000 7f4299866780 -1 Falling back to public interface 2026-02-12T18:24:31.718 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:31.714+0000 7f4299866780 -1 osd.0 0 log_to_monitors true 2026-02-12T18:24:32.446 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:24:32.446 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:24:32.447 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:24:32.447 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:24:32.447 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:24:32.447 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:24:32.674 INFO:tasks.workunit.client.0.trial116.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3649997019,v1:127.0.0.1:6803/3649997019] [v2:127.0.0.1:6804/3649997019,v1:127.0.0.1:6805/3649997019] exists,up dcd9d85f-3843-42ae-8b0e-fa874509ec83 2026-02-12T18:24:32.678 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:24:32.678 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:24:32.678 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:24:32.678 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:24:32.678 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-12T18:24:32.679 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:24:32.679 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:24:32.679 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-12T18:24:32.679 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:24:32.679 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-12T18:24:32.679 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:24:32.679 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:24:32.679 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:24:32.679 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:24:32.679 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-12T18:24:32.679 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-12T18:24:32.679 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:24:32.680 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:24:32.680 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:24:32.680 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:24:32.680 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:24:32.680 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:24:32.680 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:24:32.680 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:32.680 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:32.680 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:24:32.680 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:24:32.680 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:24:32.680 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:24:32.680 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:24:32.681 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:24:32.681 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:24:32.681 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:24:32.681 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:24:32.681 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:24:32.681 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:24:32.681 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:24:32.681 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:24:32.681 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-12T18:24:32.681 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:24:32.681 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=78ed7509-f1d6-4dae-87ac-0e7f84c8d9f1 2026-02-12T18:24:32.681 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 78ed7509-f1d6-4dae-87ac-0e7f84c8d9f1' 2026-02-12T18:24:32.682 INFO:tasks.workunit.client.0.trial116.stdout:add osd1 78ed7509-f1d6-4dae-87ac-0e7f84c8d9f1 2026-02-12T18:24:32.682 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:24:32.688 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDgGo5pF37SKBAAxiCWyN992KSEpx8pHiCghg== 2026-02-12T18:24:32.688 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDgGo5pF37SKBAAxiCWyN992KSEpx8pHiCghg=="}' 2026-02-12T18:24:32.688 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 78ed7509-f1d6-4dae-87ac-0e7f84c8d9f1 -i td/test-erasure-eio/1/new.json 2026-02-12T18:24:32.918 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:24:32.926 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-12T18:24:32.927 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDgGo5pF37SKBAAxiCWyN992KSEpx8pHiCghg== --osd-uuid 78ed7509-f1d6-4dae-87ac-0e7f84c8d9f1 2026-02-12T18:24:32.942 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:32.939+0000 7f93fa42e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:32.943 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:32.939+0000 7f93fa42e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:32.944 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:32.941+0000 7f93fa42e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:32.944 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:32.941+0000 7f93fa42e780 -1 bdev(0x31c8c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-12T18:24:32.945 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:32.941+0000 7f93fa42e780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-12T18:24:33.266 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-12T18:24:33.266 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:24:33.267 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-12T18:24:33.267 INFO:tasks.workunit.client.0.trial116.stdout:adding osd1 key to auth repository 2026-02-12T18:24:33.267 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:24:33.603 INFO:tasks.workunit.client.0.trial116.stdout:start osd.1 2026-02-12T18:24:33.604 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-12T18:24:33.604 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:24:33.604 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:24:33.604 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:24:33.604 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:24:33.618 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:33.615+0000 7ff482834780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:33.619 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:33.616+0000 7ff482834780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:33.620 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:33.617+0000 7ff482834780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:33.752 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:33.749+0000 7ff482834780 -1 Falling back to public interface 2026-02-12T18:24:33.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-12T18:24:33.829 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:24:33.829 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T18:24:33.829 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:24:33.829 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:24:33.829 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:24:33.829 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:24:33.829 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:24:33.830 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:24:33.830 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:24:34.014 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:34.011+0000 7ff482834780 -1 osd.1 0 log_to_monitors true 2026-02-12T18:24:34.055 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:24:35.056 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:24:35.056 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:24:35.057 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:24:35.057 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:24:35.058 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:24:35.058 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:24:35.284 INFO:tasks.workunit.client.0.trial116.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1495147065,v1:127.0.0.1:6811/1495147065] [v2:127.0.0.1:6812/1495147065,v1:127.0.0.1:6813/1495147065] exists,up 78ed7509-f1d6-4dae-87ac-0e7f84c8d9f1 2026-02-12T18:24:35.288 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:24:35.288 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:24:35.288 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:24:35.288 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:24:35.288 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-12T18:24:35.288 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:24:35.288 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:24:35.288 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-12T18:24:35.289 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:24:35.289 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-12T18:24:35.289 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:24:35.289 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:24:35.289 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:24:35.289 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:24:35.289 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-12T18:24:35.289 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-12T18:24:35.289 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:24:35.289 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:24:35.289 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:24:35.289 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:24:35.290 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:24:35.290 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:24:35.290 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:24:35.290 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:35.290 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:35.290 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:24:35.290 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:24:35.290 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:24:35.290 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:24:35.290 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:24:35.290 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:24:35.290 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:24:35.290 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:24:35.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:24:35.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:24:35.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:24:35.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:24:35.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:24:35.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-12T18:24:35.291 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:24:35.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f5914488-35e8-4728-ac5e-1779823dcaa0 2026-02-12T18:24:35.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 f5914488-35e8-4728-ac5e-1779823dcaa0' 2026-02-12T18:24:35.291 INFO:tasks.workunit.client.0.trial116.stdout:add osd2 f5914488-35e8-4728-ac5e-1779823dcaa0 2026-02-12T18:24:35.292 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:24:35.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDjGo5pz7GNERAAXd0CiFSroE1q5yfbeARB/w== 2026-02-12T18:24:35.298 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDjGo5pz7GNERAAXd0CiFSroE1q5yfbeARB/w=="}' 2026-02-12T18:24:35.298 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f5914488-35e8-4728-ac5e-1779823dcaa0 -i td/test-erasure-eio/2/new.json 2026-02-12T18:24:35.526 INFO:tasks.workunit.client.0.trial116.stdout:2 2026-02-12T18:24:35.533 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-12T18:24:35.534 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDjGo5pz7GNERAAXd0CiFSroE1q5yfbeARB/w== --osd-uuid f5914488-35e8-4728-ac5e-1779823dcaa0 2026-02-12T18:24:35.549 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:35.546+0000 7f78ed833780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:35.550 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:35.547+0000 7f78ed833780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:35.551 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:35.548+0000 7f78ed833780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:35.552 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:35.549+0000 7f78ed833780 -1 bdev(0x2b1ec00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-12T18:24:35.552 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:35.549+0000 7f78ed833780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-12T18:24:35.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-12T18:24:35.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:24:35.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-12T18:24:35.859 INFO:tasks.workunit.client.0.trial116.stdout:adding osd2 key to auth repository 2026-02-12T18:24:35.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:24:36.181 INFO:tasks.workunit.client.0.trial116.stdout:start osd.2 2026-02-12T18:24:36.182 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-12T18:24:36.182 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:24:36.183 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:24:36.183 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:24:36.183 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:24:36.196 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:36.193+0000 7f5e6a62d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:36.198 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:36.195+0000 7f5e6a62d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:36.198 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:36.196+0000 7f5e6a62d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:36.357 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:36.354+0000 7f5e6a62d780 -1 Falling back to public interface 2026-02-12T18:24:36.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-12T18:24:36.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:24:36.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T18:24:36.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:24:36.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:24:36.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:24:36.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:24:36.409 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:24:36.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:24:36.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:24:36.632 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:24:36.672 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:36.669+0000 7f5e6a62d780 -1 osd.2 0 log_to_monitors true 2026-02-12T18:24:37.634 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:24:37.634 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:24:37.634 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:24:37.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:24:37.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:24:37.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:24:37.867 INFO:tasks.workunit.client.0.trial116.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2156032855,v1:127.0.0.1:6819/2156032855] [v2:127.0.0.1:6820/2156032855,v1:127.0.0.1:6821/2156032855] exists,up f5914488-35e8-4728-ac5e-1779823dcaa0 2026-02-12T18:24:37.871 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:24:37.871 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:24:37.872 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:24:37.872 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:24:37.872 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-12T18:24:37.872 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:24:37.872 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:24:37.872 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-12T18:24:37.872 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:24:37.872 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-12T18:24:37.872 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:24:37.872 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:24:37.872 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:24:37.872 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:24:37.873 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-12T18:24:37.873 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-12T18:24:37.873 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:24:37.873 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:24:37.873 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:24:37.873 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:24:37.873 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:24:37.873 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:24:37.873 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:24:37.873 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:37.873 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:37.874 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:24:37.874 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:24:37.874 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:24:37.874 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:24:37.874 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:24:37.874 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:24:37.874 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:24:37.874 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:24:37.874 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:24:37.874 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:24:37.874 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:24:37.874 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:24:37.874 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:24:37.875 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-12T18:24:37.875 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:24:37.875 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e7ddce77-d33a-4d1e-9e3c-f63794b56e91 2026-02-12T18:24:37.875 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 e7ddce77-d33a-4d1e-9e3c-f63794b56e91' 2026-02-12T18:24:37.875 INFO:tasks.workunit.client.0.trial116.stdout:add osd3 e7ddce77-d33a-4d1e-9e3c-f63794b56e91 2026-02-12T18:24:37.876 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:24:37.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDlGo5psExQNBAAwtke43DXB+2Tc8baaAF21w== 2026-02-12T18:24:37.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDlGo5psExQNBAAwtke43DXB+2Tc8baaAF21w=="}' 2026-02-12T18:24:37.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e7ddce77-d33a-4d1e-9e3c-f63794b56e91 -i td/test-erasure-eio/3/new.json 2026-02-12T18:24:38.121 INFO:tasks.workunit.client.0.trial116.stdout:3 2026-02-12T18:24:38.128 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-12T18:24:38.129 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDlGo5psExQNBAAwtke43DXB+2Tc8baaAF21w== --osd-uuid e7ddce77-d33a-4d1e-9e3c-f63794b56e91 2026-02-12T18:24:38.144 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:38.141+0000 7f6e011e3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:38.145 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:38.143+0000 7f6e011e3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:38.146 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:38.143+0000 7f6e011e3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:38.147 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:38.144+0000 7f6e011e3780 -1 bdev(0x2edec00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-12T18:24:38.147 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:38.144+0000 7f6e011e3780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-12T18:24:38.476 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-12T18:24:38.477 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:24:38.477 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-12T18:24:38.477 INFO:tasks.workunit.client.0.trial116.stdout:adding osd3 key to auth repository 2026-02-12T18:24:38.478 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:24:38.793 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-12T18:24:38.794 INFO:tasks.workunit.client.0.trial116.stdout:start osd.3 2026-02-12T18:24:38.794 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:24:38.795 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:24:38.795 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:24:38.795 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:24:38.807 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:38.804+0000 7f7ccb1e3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:38.808 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:38.806+0000 7f7ccb1e3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:38.809 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:38.807+0000 7f7ccb1e3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:38.927 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:38.924+0000 7f7ccb1e3780 -1 Falling back to public interface 2026-02-12T18:24:39.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-12T18:24:39.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:24:39.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-12T18:24:39.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:24:39.029 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:24:39.030 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:24:39.030 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:24:39.030 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:24:39.030 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:24:39.031 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:24:39.228 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:39.225+0000 7f7ccb1e3780 -1 osd.3 0 log_to_monitors true 2026-02-12T18:24:39.256 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:24:40.258 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:24:40.258 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:24:40.258 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:24:40.259 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:24:40.259 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:24:40.259 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:24:40.494 INFO:tasks.workunit.client.0.trial116.stdout:osd.3 up in weight 1 up_from 27 up_thru 28 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1254743483,v1:127.0.0.1:6827/1254743483] [v2:127.0.0.1:6828/1254743483,v1:127.0.0.1:6829/1254743483] exists,up e7ddce77-d33a-4d1e-9e3c-f63794b56e91 2026-02-12T18:24:40.495 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:24:40.496 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:24:40.496 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:24:40.496 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-12T18:24:40.496 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T18:24:40.496 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T18:24:40.496 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:24:40.496 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:40.496 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:40.496 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.0.asok 2026-02-12T18:24:40.496 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-12T18:24:40.496 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-osd.0.asok log flush 2026-02-12T18:24:40.540 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:24:40.543 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-12T18:24:40.546 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:24:31.470+0000 7f4299866780 0 load: jerasure load: lrc 2026-02-12T18:24:40.548 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:278: TEST_rados_get_subread_eio_shard_0: local poolname=pool-jerasure 2026-02-12T18:24:40.548 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:279: TEST_rados_get_subread_eio_shard_0: create_erasure_coded_pool pool-jerasure 2 1 2026-02-12T18:24:40.548 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-12T18:24:40.548 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-12T18:24:40.548 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-12T18:24:40.548 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-12T18:24:40.549 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-12T18:24:40.549 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-12T18:24:40.549 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-12T18:24:40.834 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-12T18:24:40.834 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-12T18:24:41.207 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-jerasure' already exists 2026-02-12T18:24:41.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:24:42.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-12T18:24:42.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:24:42.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:24:42.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:24:42.218 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:24:42.218 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:24:42.219 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:24:42.219 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:24:42.219 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:24:42.219 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:24:42.307 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:24:42.307 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:24:42.308 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:24:42.308 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:24:42.308 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:24:42.308 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:24:42.532 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:24:42.533 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:24:42.533 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:24:42.533 INFO:tasks.workunit.client.0.trial116.stderr:3' 2026-02-12T18:24:42.533 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:24:42.533 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:24:42.533 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:24:42.596 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803780 2026-02-12T18:24:42.597 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803780 2026-02-12T18:24:42.597 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780' 2026-02-12T18:24:42.597 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:24:42.597 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:24:42.660 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607556 2026-02-12T18:24:42.661 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607556 2026-02-12T18:24:42.661 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-51539607556' 2026-02-12T18:24:42.661 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:24:42.661 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:24:42.726 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411331 2026-02-12T18:24:42.726 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411331 2026-02-12T18:24:42.726 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-51539607556 2-77309411331' 2026-02-12T18:24:42.727 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:24:42.727 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:24:42.790 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116995 2026-02-12T18:24:42.790 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116995 2026-02-12T18:24:42.790 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-51539607556 2-77309411331 3-115964116995' 2026-02-12T18:24:42.790 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:24:42.790 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803780 2026-02-12T18:24:42.791 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:24:42.791 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:24:42.792 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803780 2026-02-12T18:24:42.792 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:24:42.793 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803780 2026-02-12T18:24:42.793 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803780' 2026-02-12T18:24:42.794 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.0 seq 25769803780 2026-02-12T18:24:42.794 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:24:43.016 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803778 -lt 25769803780 2026-02-12T18:24:43.016 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:24:44.017 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:24:44.018 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:24:44.243 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803780 2026-02-12T18:24:44.243 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:24:44.244 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607556 2026-02-12T18:24:44.244 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:24:44.245 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:24:44.245 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607556 2026-02-12T18:24:44.245 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:24:44.247 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607556 2026-02-12T18:24:44.247 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607556' 2026-02-12T18:24:44.247 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.1 seq 51539607556 2026-02-12T18:24:44.247 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:24:44.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607556 -lt 51539607556 2026-02-12T18:24:44.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:24:44.473 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411331 2026-02-12T18:24:44.473 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:24:44.473 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:24:44.474 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411331 2026-02-12T18:24:44.474 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:24:44.475 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411331 2026-02-12T18:24:44.475 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.2 seq 77309411331 2026-02-12T18:24:44.476 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411331' 2026-02-12T18:24:44.476 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:24:44.701 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411332 -lt 77309411331 2026-02-12T18:24:44.701 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:24:44.701 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-115964116995 2026-02-12T18:24:44.702 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:24:44.702 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:24:44.703 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964116995 2026-02-12T18:24:44.703 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:24:44.704 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964116995 2026-02-12T18:24:44.704 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 115964116995' 2026-02-12T18:24:44.704 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.3 seq 115964116995 2026-02-12T18:24:44.705 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:24:44.927 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964116995 -lt 115964116995 2026-02-12T18:24:44.927 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:24:44.928 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:24:44.928 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:24:45.242 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:24:45.242 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:24:45.242 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:24:45.242 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:24:45.242 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:24:45.242 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:24:45.242 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:24:45.243 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:24:45.464 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:24:45.465 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:24:45.465 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:24:45.465 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:24:45.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:24:45.783 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:24:45.784 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:24:45.784 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:281: TEST_rados_get_subread_eio_shard_0: local shard_id=0 2026-02-12T18:24:45.784 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:282: TEST_rados_get_subread_eio_shard_0: rados_put_get_data eio td/test-erasure-eio 0 2026-02-12T18:24:45.784 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-12T18:24:45.784 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-12T18:24:45.784 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-12T18:24:45.784 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-12T18:24:45.785 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=0 2026-02-12T18:24:45.785 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-12T18:24:45.785 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg= 2026-02-12T18:24:45.785 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-12T18:24:45.785 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-76821-0 2026-02-12T18:24:45.785 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-76821-0 2026-02-12T18:24:45.785 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-12T18:24:45.785 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-12T18:24:45.785 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-76821-0 2026-02-12T18:24:45.785 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:24:45.785 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-12T18:24:45.785 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:24:45.786 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-12T18:24:45.786 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:24:45.786 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-12T18:24:45.786 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:24:45.786 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-12T18:24:45.786 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-76821-0 td/test-erasure-eio/ORIGINAL 2026-02-12T18:24:45.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-76821-0 td/test-erasure-eio 0 2026-02-12T18:24:45.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-12T18:24:45.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-12T18:24:45.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-12T18:24:45.801 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-12T18:24:45.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-12T18:24:45.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-12T18:24:45.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj-eio-76821-0 2026-02-12T18:24:45.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-12T18:24:45.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-12T18:24:45.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-12T18:24:45.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=0 2026-02-12T18:24:45.802 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-12T18:24:45.802 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj-eio-76821-0 2026-02-12T18:24:45.802 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:24:45.802 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-76821-0 2026-02-12T18:24:45.802 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-76821-0 2026-02-12T18:24:45.803 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:24:46.038 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:24:46.038 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:24:46.038 INFO:tasks.workunit.client.0.trial116.stderr:2' 2026-02-12T18:24:46.038 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-12T18:24:46.038 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '2') 2026-02-12T18:24:46.038 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-12T18:24:46.039 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=3 2026-02-12T18:24:46.039 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-12T18:24:46.039 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/3/type 2026-02-12T18:24:46.040 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-12T18:24:46.041 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-12T18:24:46.041 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-12T18:24:46.041 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-12T18:24:46.041 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-12T18:24:46.041 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-12T18:24:46.041 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-12T18:24:46.041 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-12T18:24:46.041 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-12T18:24:46.041 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-12T18:24:46.041 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:24:46.042 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:46.042 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:46.042 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.3.asok 2026-02-12T18:24:46.042 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.76821/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-12T18:24:46.087 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-12T18:24:46.087 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-12T18:24:46.088 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-12T18:24:46.089 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.3 2026-02-12T18:24:46.089 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-12T18:24:46.089 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-12T18:24:46.089 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:24:46.089 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:46.089 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:46.089 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.3.asok 2026-02-12T18:24:46.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-12T18:24:46.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-osd.3.asok injectdataerr pool-jerasure obj-eio-76821-0 0 2026-02-12T18:24:46.136 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-76821-0 2026-02-12T18:24:46.137 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-12T18:24:46.137 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-12T18:24:46.137 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-76821-0 2026-02-12T18:24:46.137 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-12T18:24:46.137 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-12T18:24:46.137 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-76821-0 td/test-erasure-eio/COPY 2026-02-12T18:24:46.153 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-12T18:24:46.154 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-12T18:24:46.155 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' '' = recovery ']' 2026-02-12T18:24:46.155 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: expr 0 + 1 2026-02-12T18:24:46.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: local another_shard_id=1 2026-02-12T18:24:46.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:182: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-76821-0 td/test-erasure-eio 1 2026-02-12T18:24:46.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-12T18:24:46.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-12T18:24:46.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-12T18:24:46.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-12T18:24:46.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-12T18:24:46.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-12T18:24:46.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj-eio-76821-0 2026-02-12T18:24:46.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-12T18:24:46.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-12T18:24:46.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-12T18:24:46.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=1 2026-02-12T18:24:46.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-12T18:24:46.158 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj-eio-76821-0 2026-02-12T18:24:46.158 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:24:46.158 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-76821-0 2026-02-12T18:24:46.158 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-76821-0 2026-02-12T18:24:46.158 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:24:46.387 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:24:46.387 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:24:46.387 INFO:tasks.workunit.client.0.trial116.stderr:2' 2026-02-12T18:24:46.387 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-12T18:24:46.388 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '2') 2026-02-12T18:24:46.388 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-12T18:24:46.388 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=1 2026-02-12T18:24:46.388 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-12T18:24:46.388 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/1/type 2026-02-12T18:24:46.390 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-12T18:24:46.390 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-12T18:24:46.390 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-12T18:24:46.390 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-12T18:24:46.390 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-12T18:24:46.390 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-12T18:24:46.390 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-12T18:24:46.390 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-12T18:24:46.390 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-12T18:24:46.390 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-12T18:24:46.391 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:24:46.391 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:46.391 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:46.391 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.1.asok 2026-02-12T18:24:46.391 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.76821/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-12T18:24:46.440 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-12T18:24:46.440 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-12T18:24:46.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-12T18:24:46.442 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.1 2026-02-12T18:24:46.442 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-12T18:24:46.442 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-12T18:24:46.442 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:24:46.442 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:46.442 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:46.442 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.1.asok 2026-02-12T18:24:46.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-12T18:24:46.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-osd.1.asok injectdataerr pool-jerasure obj-eio-76821-0 1 2026-02-12T18:24:46.493 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:183: rados_put_get_data: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-12T18:24:46.493 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:192: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-76821-0 fail 2026-02-12T18:24:46.493 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-12T18:24:46.493 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-12T18:24:46.493 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-76821-0 2026-02-12T18:24:46.493 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-12T18:24:46.493 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-12T18:24:46.493 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-eio-76821-0 td/test-erasure-eio/COPY 2026-02-12T18:24:46.508 INFO:tasks.workunit.client.0.trial116.stderr:error getting pool-jerasure/obj-eio-76821-0: (5) Input/output error 2026-02-12T18:24:46.509 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-12T18:24:46.510 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: rados_put_get_data: rm td/test-erasure-eio/ORIGINAL 2026-02-12T18:24:46.510 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:283: TEST_rados_get_subread_eio_shard_0: delete_erasure_coded_pool pool-jerasure 2026-02-12T18:24:46.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-12T18:24:46.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-12T18:24:46.787 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-jerasure' does not exist 2026-02-12T18:24:46.794 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-12T18:24:47.071 INFO:tasks.workunit.client.0.trial116.stderr:erasure-code-profile myprofile does not exist 2026-02-12T18:24:47.080 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-12T18:24:47.080 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-12T18:24:47.080 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:24:47.080 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-12T18:24:47.081 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:24:47.081 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:24:47.081 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:24:47.081 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:24:47.081 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:24:47.190 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:24:47.190 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:24:47.191 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:24:47.192 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:24:47.193 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:24:47.193 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:24:47.193 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:24:47.194 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:24:47.194 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:24:47.195 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:24:47.195 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:24:47.195 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:24:47.197 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:24:47.197 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-12T18:24:47.206 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:24:47.206 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:47.206 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:47.206 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76821 2026-02-12T18:24:47.208 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:24:47.208 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:24:47.208 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-12T18:24:47.208 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-12T18:24:47.209 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-12T18:24:47.209 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-12T18:24:47.209 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-12T18:24:47.209 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:24:47.209 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-12T18:24:47.209 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:24:47.209 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:24:47.209 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:24:47.209 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:24:47.209 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:24:47.210 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:24:47.210 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:24:47.211 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:24:47.211 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:24:47.212 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:24:47.212 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:24:47.212 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:24:47.213 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:24:47.213 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:24:47.214 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:24:47.214 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:24:47.214 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:24:47.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:24:47.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-12T18:24:47.217 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:24:47.217 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:47.217 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:47.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76821 2026-02-12T18:24:47.217 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:24:47.218 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:24:47.218 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-12T18:24:47.219 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T18:24:47.219 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:47.219 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:47.219 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76821 2026-02-12T18:24:47.220 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T18:24:47.221 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T18:24:47.221 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T18:24:47.221 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-12T18:24:47.221 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-12T18:24:47.222 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-12T18:24:47.222 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T18:24:47.222 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T18:24:47.222 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T18:24:47.222 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-12T18:24:47.222 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-12T18:24:47.239 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T18:24:47.239 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:24:47.239 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:24:47.240 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:24:47.240 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:47.240 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:47.240 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:24:47.240 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T18:24:47.263 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T18:24:47.264 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T18:24:47.264 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:24:47.264 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:24:47.264 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T18:24:47.264 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T18:24:47.266 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:24:47.266 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:24:47.266 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:24:47.266 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:24:47.266 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:47.267 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:47.267 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:24:47.267 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:24:47.267 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76821/ceph-mon.a.asok config get fsid 2026-02-12T18:24:47.307 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T18:24:47.307 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:24:47.308 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:24:47.308 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T18:24:47.308 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T18:24:47.309 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:24:47.309 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:24:47.309 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:24:47.309 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:24:47.309 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:47.309 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:47.309 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:24:47.310 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:24:47.310 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76821/ceph-mon.a.asok config get mon_host 2026-02-12T18:24:47.351 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-12T18:24:47.351 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-12T18:24:47.351 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T18:24:47.351 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T18:24:47.351 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T18:24:47.352 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-12T18:24:47.352 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T18:24:47.450 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T18:24:47.450 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:24:47.450 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:24:47.450 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:24:47.450 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:47.450 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:47.450 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:24:47.451 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:24:47.451 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:24:47.466 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-12T18:24:47.466 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T18:24:47.559 INFO:tasks.workunit.client.0.trial116.stderr:pool 'rbd' already exists 2026-02-12T18:24:47.565 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:24:48.568 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-12T18:24:48.568 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:24:48.568 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:24:48.568 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:24:48.569 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:48.569 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:48.569 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:24:48.569 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-12T18:24:48.569 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-mon.a.asok log flush 2026-02-12T18:24:48.607 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:24:48.611 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-12T18:24:48.613 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:24:47.252+0000 7f6bb87fdc00 0 load: jerasure load: lrc 2026-02-12T18:24:48.613 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_subread_eio_shard_1 td/test-erasure-eio 2026-02-12T18:24:48.613 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:287: TEST_rados_get_subread_eio_shard_1: local dir=td/test-erasure-eio 2026-02-12T18:24:48.614 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:288: TEST_rados_get_subread_eio_shard_1: setup_osds 4 2026-02-12T18:24:48.614 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-12T18:24:48.614 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-12T18:24:48.614 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-12T18:24:48.614 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-12T18:24:48.618 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:24:48.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-12T18:24:48.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:24:48.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:24:48.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T18:24:48.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:24:48.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-12T18:24:48.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:24:48.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:24:48.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:24:48.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:24:48.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-12T18:24:48.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-12T18:24:48.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:24:48.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:24:48.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:24:48.620 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:24:48.620 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:24:48.620 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:24:48.620 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:24:48.620 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:48.620 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:48.620 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:24:48.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:24:48.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:24:48.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:24:48.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:24:48.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:24:48.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:24:48.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:24:48.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:24:48.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:24:48.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:24:48.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:24:48.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:24:48.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-12T18:24:48.621 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:24:48.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=eb285672-f994-4ef4-90d6-48f1f06883d0 2026-02-12T18:24:48.622 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 eb285672-f994-4ef4-90d6-48f1f06883d0' 2026-02-12T18:24:48.622 INFO:tasks.workunit.client.0.trial116.stdout:add osd0 eb285672-f994-4ef4-90d6-48f1f06883d0 2026-02-12T18:24:48.622 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:24:48.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDwGo5pgXBIJRAACdcEkM3gd/bGvUxvpbY6uA== 2026-02-12T18:24:48.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDwGo5pgXBIJRAACdcEkM3gd/bGvUxvpbY6uA=="}' 2026-02-12T18:24:48.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new eb285672-f994-4ef4-90d6-48f1f06883d0 -i td/test-erasure-eio/0/new.json 2026-02-12T18:24:48.731 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:24:48.737 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-12T18:24:48.738 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDwGo5pgXBIJRAACdcEkM3gd/bGvUxvpbY6uA== --osd-uuid eb285672-f994-4ef4-90d6-48f1f06883d0 2026-02-12T18:24:48.754 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:48.750+0000 7f760f251780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:48.755 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:48.752+0000 7f760f251780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:48.756 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:48.753+0000 7f760f251780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:48.757 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:48.753+0000 7f760f251780 -1 bdev(0x2556c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-12T18:24:48.757 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:48.753+0000 7f760f251780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-12T18:24:49.178 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-12T18:24:49.178 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:24:49.179 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T18:24:49.179 INFO:tasks.workunit.client.0.trial116.stdout:adding osd0 key to auth repository 2026-02-12T18:24:49.179 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:24:49.283 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T18:24:49.284 INFO:tasks.workunit.client.0.trial116.stdout:start osd.0 2026-02-12T18:24:49.284 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:24:49.285 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:24:49.285 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:24:49.285 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:24:49.298 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:49.294+0000 7f923d638780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:49.299 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:49.295+0000 7f923d638780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:49.300 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:49.296+0000 7f923d638780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:49.383 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T18:24:49.383 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:24:49.383 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T18:24:49.383 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:24:49.383 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:24:49.383 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:24:49.384 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:24:49.384 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:24:49.384 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:24:49.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:24:49.480 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:24:49.508 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:49.505+0000 7f923d638780 -1 Falling back to public interface 2026-02-12T18:24:49.785 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:49.781+0000 7f923d638780 -1 osd.0 0 log_to_monitors true 2026-02-12T18:24:50.481 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:24:50.481 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:24:50.481 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:24:50.481 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:24:50.481 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:24:50.481 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:24:50.710 INFO:tasks.workunit.client.0.trial116.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/609733688,v1:127.0.0.1:6803/609733688] [v2:127.0.0.1:6804/609733688,v1:127.0.0.1:6805/609733688] exists,up eb285672-f994-4ef4-90d6-48f1f06883d0 2026-02-12T18:24:50.714 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:24:50.714 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:24:50.715 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:24:50.715 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:24:50.715 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-12T18:24:50.715 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:24:50.715 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:24:50.715 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-12T18:24:50.715 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:24:50.715 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-12T18:24:50.715 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:24:50.715 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:24:50.715 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:24:50.715 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:24:50.716 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-12T18:24:50.716 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-12T18:24:50.716 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:24:50.716 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:24:50.716 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:24:50.716 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:24:50.716 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:24:50.716 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:24:50.716 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:24:50.716 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:50.716 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:50.716 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:24:50.717 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:24:50.717 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:24:50.717 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:24:50.717 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:24:50.717 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:24:50.717 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:24:50.717 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:24:50.717 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:24:50.717 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:24:50.717 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:24:50.717 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:24:50.717 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:24:50.717 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-12T18:24:50.718 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:24:50.718 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2166663f-d9d2-46a0-8841-f4e997257651 2026-02-12T18:24:50.718 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 2166663f-d9d2-46a0-8841-f4e997257651' 2026-02-12T18:24:50.718 INFO:tasks.workunit.client.0.trial116.stdout:add osd1 2166663f-d9d2-46a0-8841-f4e997257651 2026-02-12T18:24:50.718 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:24:50.725 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDyGo5pJlMCKxAA+w/w1WyG9FCirDkSlo3LqQ== 2026-02-12T18:24:50.725 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDyGo5pJlMCKxAA+w/w1WyG9FCirDkSlo3LqQ=="}' 2026-02-12T18:24:50.725 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2166663f-d9d2-46a0-8841-f4e997257651 -i td/test-erasure-eio/1/new.json 2026-02-12T18:24:50.956 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:24:50.963 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-12T18:24:50.964 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDyGo5pJlMCKxAA+w/w1WyG9FCirDkSlo3LqQ== --osd-uuid 2166663f-d9d2-46a0-8841-f4e997257651 2026-02-12T18:24:50.980 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:50.976+0000 7f9b03665780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:50.981 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:50.977+0000 7f9b03665780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:50.982 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:50.978+0000 7f9b03665780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:50.982 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:50.978+0000 7f9b03665780 -1 bdev(0x37e8c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-12T18:24:50.983 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:50.979+0000 7f9b03665780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-12T18:24:51.311 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-12T18:24:51.311 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:24:51.312 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-12T18:24:51.312 INFO:tasks.workunit.client.0.trial116.stdout:adding osd1 key to auth repository 2026-02-12T18:24:51.313 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:24:51.634 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-12T18:24:51.635 INFO:tasks.workunit.client.0.trial116.stdout:start osd.1 2026-02-12T18:24:51.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:24:51.635 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:24:51.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:24:51.636 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:24:51.648 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:51.644+0000 7f6a05fe3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:51.649 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:51.645+0000 7f6a05fe3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:51.650 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:51.646+0000 7f6a05fe3780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:51.809 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:51.805+0000 7f6a05fe3780 -1 Falling back to public interface 2026-02-12T18:24:51.863 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-12T18:24:51.863 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:24:51.863 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T18:24:51.864 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:24:51.864 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:24:51.864 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:24:51.864 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:24:51.864 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:24:51.865 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:24:51.865 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:24:52.088 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:24:52.185 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:52.181+0000 7f6a05fe3780 -1 osd.1 0 log_to_monitors true 2026-02-12T18:24:53.089 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:24:53.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:24:53.090 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:24:53.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:24:53.091 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:24:53.091 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:24:53.319 INFO:tasks.workunit.client.0.trial116.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1397704726,v1:127.0.0.1:6811/1397704726] [v2:127.0.0.1:6812/1397704726,v1:127.0.0.1:6813/1397704726] exists,up 2166663f-d9d2-46a0-8841-f4e997257651 2026-02-12T18:24:53.323 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:24:53.324 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:24:53.324 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:24:53.324 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:24:53.324 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-12T18:24:53.324 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:24:53.324 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:24:53.324 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-12T18:24:53.324 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:24:53.324 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-12T18:24:53.325 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:24:53.325 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:24:53.325 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:24:53.325 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:24:53.325 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-12T18:24:53.325 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-12T18:24:53.325 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:24:53.325 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:24:53.325 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:24:53.325 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:24:53.325 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:24:53.326 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:24:53.326 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:24:53.326 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:53.326 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:53.326 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:24:53.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:24:53.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:24:53.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:24:53.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:24:53.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:24:53.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:24:53.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:24:53.326 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:24:53.327 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:24:53.327 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:24:53.327 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:24:53.327 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:24:53.327 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-12T18:24:53.327 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:24:53.327 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4cf742f2-4770-49b4-ba9a-adb3708f87f9 2026-02-12T18:24:53.327 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 4cf742f2-4770-49b4-ba9a-adb3708f87f9' 2026-02-12T18:24:53.327 INFO:tasks.workunit.client.0.trial116.stdout:add osd2 4cf742f2-4770-49b4-ba9a-adb3708f87f9 2026-02-12T18:24:53.328 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:24:53.334 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD1Go5pwK6xExAAQEKo9nGy/BvtkjrrRC7obA== 2026-02-12T18:24:53.334 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD1Go5pwK6xExAAQEKo9nGy/BvtkjrrRC7obA=="}' 2026-02-12T18:24:53.334 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4cf742f2-4770-49b4-ba9a-adb3708f87f9 -i td/test-erasure-eio/2/new.json 2026-02-12T18:24:53.568 INFO:tasks.workunit.client.0.trial116.stdout:2 2026-02-12T18:24:53.576 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-12T18:24:53.577 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD1Go5pwK6xExAAQEKo9nGy/BvtkjrrRC7obA== --osd-uuid 4cf742f2-4770-49b4-ba9a-adb3708f87f9 2026-02-12T18:24:53.593 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:53.589+0000 7fd117020780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:53.595 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:53.591+0000 7fd117020780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:53.596 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:53.592+0000 7fd117020780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:53.596 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:53.592+0000 7fd117020780 -1 bdev(0x2bf0c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-12T18:24:53.597 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:53.592+0000 7fd117020780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-12T18:24:53.910 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-12T18:24:53.910 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:24:53.911 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-12T18:24:53.911 INFO:tasks.workunit.client.0.trial116.stdout:adding osd2 key to auth repository 2026-02-12T18:24:53.912 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:24:54.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-12T18:24:54.250 INFO:tasks.workunit.client.0.trial116.stdout:start osd.2 2026-02-12T18:24:54.251 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:24:54.251 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:24:54.251 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:24:54.251 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:24:54.264 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:54.260+0000 7fbbe4e08780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:54.265 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:54.261+0000 7fbbe4e08780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:54.266 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:54.262+0000 7fbbe4e08780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:54.459 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:54.455+0000 7fbbe4e08780 -1 Falling back to public interface 2026-02-12T18:24:54.475 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-12T18:24:54.475 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:24:54.476 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T18:24:54.476 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:24:54.476 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:24:54.476 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:24:54.476 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:24:54.477 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:24:54.477 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:24:54.478 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:24:54.699 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:24:54.815 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:54.811+0000 7fbbe4e08780 -1 osd.2 0 log_to_monitors true 2026-02-12T18:24:55.700 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:24:55.700 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:24:55.700 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:24:55.701 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:24:55.701 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:24:55.701 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:24:55.929 INFO:tasks.workunit.client.0.trial116.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1479324551,v1:127.0.0.1:6819/1479324551] [v2:127.0.0.1:6820/1479324551,v1:127.0.0.1:6821/1479324551] exists,up 4cf742f2-4770-49b4-ba9a-adb3708f87f9 2026-02-12T18:24:55.934 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:24:55.934 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:24:55.934 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:24:55.934 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:24:55.934 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-12T18:24:55.934 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:24:55.934 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:24:55.934 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-12T18:24:55.935 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:24:55.935 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-12T18:24:55.935 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:24:55.935 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:24:55.935 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:24:55.935 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:24:55.935 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-12T18:24:55.935 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-12T18:24:55.935 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:24:55.935 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:24:55.935 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:24:55.935 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:24:55.935 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:24:55.936 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:24:55.936 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:24:55.936 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:55.936 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:55.936 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:24:55.936 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:24:55.936 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:24:55.936 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:24:55.936 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:24:55.936 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:24:55.936 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:24:55.936 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:24:55.937 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:24:55.937 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:24:55.937 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:24:55.937 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:24:55.937 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:24:55.937 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-12T18:24:55.937 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:24:55.937 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7b226c0e-b5a9-4287-bfe8-a7a5bcd91a40 2026-02-12T18:24:55.937 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 7b226c0e-b5a9-4287-bfe8-a7a5bcd91a40' 2026-02-12T18:24:55.937 INFO:tasks.workunit.client.0.trial116.stdout:add osd3 7b226c0e-b5a9-4287-bfe8-a7a5bcd91a40 2026-02-12T18:24:55.938 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:24:55.943 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD3Go5pdtr/NxAA19jegivjRUBDfkYmM4qCKQ== 2026-02-12T18:24:55.943 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD3Go5pdtr/NxAA19jegivjRUBDfkYmM4qCKQ=="}' 2026-02-12T18:24:55.943 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7b226c0e-b5a9-4287-bfe8-a7a5bcd91a40 -i td/test-erasure-eio/3/new.json 2026-02-12T18:24:56.171 INFO:tasks.workunit.client.0.trial116.stdout:3 2026-02-12T18:24:56.179 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-12T18:24:56.180 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD3Go5pdtr/NxAA19jegivjRUBDfkYmM4qCKQ== --osd-uuid 7b226c0e-b5a9-4287-bfe8-a7a5bcd91a40 2026-02-12T18:24:56.195 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:56.192+0000 7fc84a041780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:56.197 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:56.193+0000 7fc84a041780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:56.198 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:56.194+0000 7fc84a041780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:56.198 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:56.194+0000 7fc84a041780 -1 bdev(0x3ed8c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-12T18:24:56.199 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:56.195+0000 7fc84a041780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-12T18:24:56.527 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-12T18:24:56.527 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:24:56.528 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-12T18:24:56.528 INFO:tasks.workunit.client.0.trial116.stdout:adding osd3 key to auth repository 2026-02-12T18:24:56.529 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:24:56.866 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-12T18:24:56.866 INFO:tasks.workunit.client.0.trial116.stdout:start osd.3 2026-02-12T18:24:56.867 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:24:56.867 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:24:56.867 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:24:56.867 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:24:56.880 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:56.876+0000 7fac61a55780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:56.881 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:56.877+0000 7fac61a55780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:56.882 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:56.878+0000 7fac61a55780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:24:57.025 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:57.022+0000 7fac61a55780 -1 Falling back to public interface 2026-02-12T18:24:57.093 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-12T18:24:57.093 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:24:57.093 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-12T18:24:57.093 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:24:57.093 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:24:57.093 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:24:57.094 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:24:57.094 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:24:57.094 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:24:57.094 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:24:57.318 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:24:57.384 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:24:57.381+0000 7fac61a55780 -1 osd.3 0 log_to_monitors true 2026-02-12T18:24:58.319 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:24:58.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:24:58.320 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:24:58.320 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:24:58.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:24:58.321 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:24:58.548 INFO:tasks.workunit.client.0.trial116.stdout:osd.3 up in weight 1 up_from 27 up_thru 28 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3304836684,v1:127.0.0.1:6827/3304836684] [v2:127.0.0.1:6828/3304836684,v1:127.0.0.1:6829/3304836684] exists,up 7b226c0e-b5a9-4287-bfe8-a7a5bcd91a40 2026-02-12T18:24:58.550 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:24:58.550 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:24:58.550 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:24:58.550 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-12T18:24:58.550 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T18:24:58.550 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T18:24:58.550 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:24:58.550 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:24:58.550 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:24:58.551 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.0.asok 2026-02-12T18:24:58.551 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-12T18:24:58.551 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-osd.0.asok log flush 2026-02-12T18:24:58.595 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:24:58.599 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-12T18:24:58.602 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:24:49.507+0000 7f923d638780 0 load: jerasure load: lrc 2026-02-12T18:24:58.603 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:290: TEST_rados_get_subread_eio_shard_1: local poolname=pool-jerasure 2026-02-12T18:24:58.603 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:291: TEST_rados_get_subread_eio_shard_1: create_erasure_coded_pool pool-jerasure 2 1 2026-02-12T18:24:58.603 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-12T18:24:58.603 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-12T18:24:58.603 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-12T18:24:58.603 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-12T18:24:58.603 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-12T18:24:58.603 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-12T18:24:58.603 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-12T18:24:58.889 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-12T18:24:58.889 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-12T18:24:59.264 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-jerasure' already exists 2026-02-12T18:24:59.272 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:25:00.274 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-12T18:25:00.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:25:00.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:25:00.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:25:00.276 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:25:00.276 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:25:00.276 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:25:00.276 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:25:00.276 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:25:00.276 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:25:00.364 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:25:00.365 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:25:00.365 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:25:00.365 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:25:00.365 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:25:00.365 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:25:00.588 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:25:00.588 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:25:00.589 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:25:00.589 INFO:tasks.workunit.client.0.trial116.stderr:3' 2026-02-12T18:25:00.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:25:00.589 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:25:00.589 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:25:00.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803781 2026-02-12T18:25:00.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803781 2026-02-12T18:25:00.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781' 2026-02-12T18:25:00.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:25:00.651 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:25:00.716 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607556 2026-02-12T18:25:00.716 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607556 2026-02-12T18:25:00.716 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607556' 2026-02-12T18:25:00.716 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:25:00.716 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:25:00.779 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411332 2026-02-12T18:25:00.780 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411332 2026-02-12T18:25:00.780 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607556 2-77309411332' 2026-02-12T18:25:00.780 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:25:00.780 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:25:00.843 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116995 2026-02-12T18:25:00.843 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116995 2026-02-12T18:25:00.843 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803781 1-51539607556 2-77309411332 3-115964116995' 2026-02-12T18:25:00.843 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:25:00.844 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803781 2026-02-12T18:25:00.844 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:25:00.844 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:25:00.845 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803781 2026-02-12T18:25:00.845 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:25:00.846 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803781 2026-02-12T18:25:00.846 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-12T18:25:00.847 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.0 seq 25769803781 2026-02-12T18:25:00.847 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:25:01.071 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803779 -lt 25769803781 2026-02-12T18:25:01.071 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:25:02.072 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:25:02.072 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:25:02.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803781 2026-02-12T18:25:02.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:25:02.298 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607556 2026-02-12T18:25:02.298 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:25:02.299 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:25:02.299 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607556 2026-02-12T18:25:02.299 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:25:02.300 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607556 2026-02-12T18:25:02.301 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607556' 2026-02-12T18:25:02.301 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.1 seq 51539607556 2026-02-12T18:25:02.301 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:25:02.523 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607556 -lt 51539607556 2026-02-12T18:25:02.523 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:25:02.524 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411332 2026-02-12T18:25:02.524 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:25:02.525 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:25:02.526 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411332 2026-02-12T18:25:02.526 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:25:02.527 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411332 2026-02-12T18:25:02.527 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411332' 2026-02-12T18:25:02.528 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.2 seq 77309411332 2026-02-12T18:25:02.528 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:25:02.752 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411333 -lt 77309411332 2026-02-12T18:25:02.752 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:25:02.753 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-115964116995 2026-02-12T18:25:02.753 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:25:02.753 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:25:02.754 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964116995 2026-02-12T18:25:02.754 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:25:02.756 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=115964116995 2026-02-12T18:25:02.756 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 115964116995' 2026-02-12T18:25:02.756 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.3 seq 115964116995 2026-02-12T18:25:02.756 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:25:02.992 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 115964116995 -lt 115964116995 2026-02-12T18:25:02.993 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:25:02.993 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:25:02.993 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:25:03.312 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:25:03.312 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:25:03.312 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:25:03.313 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:25:03.313 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:25:03.313 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:25:03.313 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:25:03.313 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:25:03.538 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:25:03.539 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:25:03.539 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:25:03.539 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:25:03.857 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:25:03.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:25:03.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:25:03.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:293: TEST_rados_get_subread_eio_shard_1: local shard_id=1 2026-02-12T18:25:03.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:294: TEST_rados_get_subread_eio_shard_1: rados_put_get_data eio td/test-erasure-eio 1 2026-02-12T18:25:03.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-12T18:25:03.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-12T18:25:03.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-12T18:25:03.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-12T18:25:03.858 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=1 2026-02-12T18:25:03.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-12T18:25:03.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg= 2026-02-12T18:25:03.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-12T18:25:03.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-76821-1 2026-02-12T18:25:03.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-76821-1 2026-02-12T18:25:03.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-12T18:25:03.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-12T18:25:03.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-76821-1 2026-02-12T18:25:03.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:25:03.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-12T18:25:03.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:25:03.859 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-12T18:25:03.860 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:25:03.860 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-12T18:25:03.860 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:25:03.860 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-12T18:25:03.860 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-76821-1 td/test-erasure-eio/ORIGINAL 2026-02-12T18:25:03.875 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-76821-1 td/test-erasure-eio 1 2026-02-12T18:25:03.875 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-12T18:25:03.875 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-12T18:25:03.876 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-12T18:25:03.876 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-12T18:25:03.876 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-12T18:25:03.876 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-12T18:25:03.876 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj-eio-76821-1 2026-02-12T18:25:03.876 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-12T18:25:03.876 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-12T18:25:03.876 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-12T18:25:03.876 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=1 2026-02-12T18:25:03.876 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-12T18:25:03.876 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj-eio-76821-1 2026-02-12T18:25:03.876 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:25:03.877 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-76821-1 2026-02-12T18:25:03.877 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-76821-1 2026-02-12T18:25:03.877 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:25:04.113 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:25:04.114 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:25:04.114 INFO:tasks.workunit.client.0.trial116.stderr:2' 2026-02-12T18:25:04.114 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-12T18:25:04.114 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '2') 2026-02-12T18:25:04.114 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-12T18:25:04.114 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=1 2026-02-12T18:25:04.114 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-12T18:25:04.114 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/1/type 2026-02-12T18:25:04.116 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-12T18:25:04.116 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-12T18:25:04.118 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-12T18:25:04.118 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-12T18:25:04.118 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-12T18:25:04.119 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-12T18:25:04.119 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-12T18:25:04.119 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-12T18:25:04.119 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-12T18:25:04.119 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-12T18:25:04.119 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:25:04.119 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:04.119 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:04.119 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.1.asok 2026-02-12T18:25:04.119 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.76821/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-12T18:25:04.165 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-12T18:25:04.165 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-12T18:25:04.166 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-12T18:25:04.166 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.1 2026-02-12T18:25:04.166 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-12T18:25:04.166 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-12T18:25:04.166 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:25:04.166 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:04.166 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:04.166 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.1.asok 2026-02-12T18:25:04.167 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-12T18:25:04.167 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-osd.1.asok injectdataerr pool-jerasure obj-eio-76821-1 1 2026-02-12T18:25:04.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-76821-1 2026-02-12T18:25:04.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-12T18:25:04.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-12T18:25:04.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-76821-1 2026-02-12T18:25:04.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-12T18:25:04.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-12T18:25:04.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-76821-1 td/test-erasure-eio/COPY 2026-02-12T18:25:04.232 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-12T18:25:04.233 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-12T18:25:04.234 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' '' = recovery ']' 2026-02-12T18:25:04.234 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: expr 1 + 1 2026-02-12T18:25:04.237 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: local another_shard_id=2 2026-02-12T18:25:04.238 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:182: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-76821-1 td/test-erasure-eio 2 2026-02-12T18:25:04.238 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-12T18:25:04.238 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-12T18:25:04.238 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-12T18:25:04.238 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-12T18:25:04.238 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-12T18:25:04.238 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-12T18:25:04.238 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj-eio-76821-1 2026-02-12T18:25:04.238 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-12T18:25:04.239 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-12T18:25:04.239 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-12T18:25:04.239 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=2 2026-02-12T18:25:04.239 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-12T18:25:04.239 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj-eio-76821-1 2026-02-12T18:25:04.239 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:25:04.239 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-76821-1 2026-02-12T18:25:04.239 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-76821-1 2026-02-12T18:25:04.239 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:25:04.464 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:25:04.464 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:25:04.464 INFO:tasks.workunit.client.0.trial116.stderr:2' 2026-02-12T18:25:04.464 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-12T18:25:04.464 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '2') 2026-02-12T18:25:04.464 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-12T18:25:04.464 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=2 2026-02-12T18:25:04.464 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-12T18:25:04.464 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/2/type 2026-02-12T18:25:04.465 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-12T18:25:04.466 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 2 bluestore_debug_inject_read_err true 2026-02-12T18:25:04.466 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-12T18:25:04.466 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=2 2026-02-12T18:25:04.466 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-12T18:25:04.466 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-12T18:25:04.467 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-12T18:25:04.467 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.2 2026-02-12T18:25:04.467 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-12T18:25:04.468 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-12T18:25:04.468 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:25:04.468 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:04.468 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:04.468 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.2.asok 2026-02-12T18:25:04.468 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.76821/ceph-osd.2.asok config set bluestore_debug_inject_read_err true 2026-02-12T18:25:04.516 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-12T18:25:04.517 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-12T18:25:04.517 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-12T18:25:04.517 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.2 2026-02-12T18:25:04.518 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-12T18:25:04.518 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-12T18:25:04.519 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:25:04.519 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:04.519 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:04.519 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.2.asok 2026-02-12T18:25:04.519 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-12T18:25:04.519 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-osd.2.asok injectdataerr pool-jerasure obj-eio-76821-1 2 2026-02-12T18:25:04.568 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:183: rados_put_get_data: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-12T18:25:04.568 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:188: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-76821-1 2026-02-12T18:25:04.568 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-12T18:25:04.568 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-12T18:25:04.569 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-76821-1 2026-02-12T18:25:04.569 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-12T18:25:04.569 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-12T18:25:04.569 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-76821-1 td/test-erasure-eio/COPY 2026-02-12T18:25:04.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-12T18:25:04.585 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-12T18:25:04.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: rados_put_get_data: rm td/test-erasure-eio/ORIGINAL 2026-02-12T18:25:04.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:295: TEST_rados_get_subread_eio_shard_1: delete_erasure_coded_pool pool-jerasure 2026-02-12T18:25:04.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-12T18:25:04.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-12T18:25:04.865 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-jerasure' does not exist 2026-02-12T18:25:04.873 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-12T18:25:05.156 INFO:tasks.workunit.client.0.trial116.stderr:erasure-code-profile myprofile does not exist 2026-02-12T18:25:05.164 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-12T18:25:05.165 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-12T18:25:05.165 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:25:05.165 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-12T18:25:05.165 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:25:05.165 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:25:05.166 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:25:05.166 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:25:05.166 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:25:05.273 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:25:05.273 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:25:05.274 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:25:05.274 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:25:05.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:25:05.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:25:05.276 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:25:05.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:25:05.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:25:05.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:25:05.277 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:25:05.278 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:25:05.279 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:25:05.279 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-12T18:25:05.288 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:25:05.289 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:05.289 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:05.289 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76821 2026-02-12T18:25:05.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:25:05.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:25:05.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-12T18:25:05.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-12T18:25:05.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-12T18:25:05.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-12T18:25:05.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-12T18:25:05.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:25:05.291 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-12T18:25:05.291 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:25:05.291 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:25:05.292 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:25:05.292 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:25:05.292 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:25:05.292 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:25:05.292 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:25:05.293 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:25:05.294 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:25:05.295 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:25:05.295 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:25:05.295 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:25:05.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:25:05.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:25:05.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:25:05.297 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:25:05.297 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:25:05.298 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:25:05.298 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-12T18:25:05.299 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:25:05.300 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:05.300 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:05.300 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76821 2026-02-12T18:25:05.300 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:25:05.301 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:25:05.301 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-12T18:25:05.302 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T18:25:05.302 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:05.302 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:05.302 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76821 2026-02-12T18:25:05.303 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T18:25:05.304 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T18:25:05.304 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T18:25:05.304 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-12T18:25:05.305 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-12T18:25:05.305 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-12T18:25:05.305 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T18:25:05.305 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T18:25:05.305 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T18:25:05.305 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-12T18:25:05.305 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-12T18:25:05.322 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T18:25:05.322 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:25:05.322 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:25:05.322 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:25:05.322 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:05.323 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:05.323 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:25:05.323 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T18:25:05.346 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T18:25:05.347 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T18:25:05.348 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:25:05.348 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:25:05.348 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T18:25:05.348 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T18:25:05.349 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:25:05.349 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:25:05.349 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:25:05.349 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:25:05.350 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:05.350 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:05.350 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:25:05.350 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:25:05.350 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76821/ceph-mon.a.asok config get fsid 2026-02-12T18:25:05.392 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T18:25:05.392 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:25:05.392 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:25:05.392 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T18:25:05.392 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T18:25:05.393 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:25:05.393 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:25:05.393 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:25:05.394 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:25:05.394 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:05.394 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:05.394 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:25:05.394 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:25:05.394 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76821/ceph-mon.a.asok config get mon_host 2026-02-12T18:25:05.435 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-12T18:25:05.436 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-12T18:25:05.436 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T18:25:05.436 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T18:25:05.436 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T18:25:05.436 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-12T18:25:05.436 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T18:25:05.537 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T18:25:05.537 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:25:05.537 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:25:05.537 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:25:05.537 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:05.537 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:05.537 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:25:05.538 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:25:05.538 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:25:05.553 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-12T18:25:05.554 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T18:25:05.648 INFO:tasks.workunit.client.0.trial116.stderr:pool 'rbd' already exists 2026-02-12T18:25:05.654 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:25:06.657 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-12T18:25:06.657 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:25:06.657 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:25:06.657 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:25:06.657 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:06.657 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:06.657 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:25:06.658 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-12T18:25:06.658 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-mon.a.asok log flush 2026-02-12T18:25:06.696 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:25:06.700 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-12T18:25:06.701 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:25:05.336+0000 7f0f37c71c00 0 load: jerasure load: lrc 2026-02-12T18:25:06.702 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_subread_missing td/test-erasure-eio 2026-02-12T18:25:06.702 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:302: TEST_rados_get_subread_missing: local dir=td/test-erasure-eio 2026-02-12T18:25:06.702 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:303: TEST_rados_get_subread_missing: setup_osds 4 2026-02-12T18:25:06.702 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-12T18:25:06.703 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-12T18:25:06.703 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-12T18:25:06.703 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-12T18:25:06.707 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:25:06.707 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-12T18:25:06.707 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:25:06.707 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:25:06.707 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T18:25:06.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:25:06.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-12T18:25:06.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:25:06.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:25:06.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:25:06.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:25:06.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-12T18:25:06.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-12T18:25:06.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:25:06.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:25:06.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:25:06.708 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:25:06.709 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:25:06.709 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:25:06.709 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:25:06.709 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:06.709 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:06.709 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:25:06.709 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:25:06.709 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:25:06.709 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:25:06.709 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:25:06.709 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:25:06.709 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:25:06.710 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:25:06.710 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:25:06.710 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:25:06.710 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:25:06.710 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:25:06.710 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:25:06.710 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-12T18:25:06.710 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:25:06.710 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=af288f34-dfe7-42f0-8ea0-bc7e9610fa17 2026-02-12T18:25:06.710 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 af288f34-dfe7-42f0-8ea0-bc7e9610fa17' 2026-02-12T18:25:06.711 INFO:tasks.workunit.client.0.trial116.stdout:add osd0 af288f34-dfe7-42f0-8ea0-bc7e9610fa17 2026-02-12T18:25:06.711 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:25:06.718 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQACG45pndKWKhAAqV+JKZ54KVP7kABF/obBrg== 2026-02-12T18:25:06.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQACG45pndKWKhAAqV+JKZ54KVP7kABF/obBrg=="}' 2026-02-12T18:25:06.719 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new af288f34-dfe7-42f0-8ea0-bc7e9610fa17 -i td/test-erasure-eio/0/new.json 2026-02-12T18:25:06.819 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:25:06.825 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-12T18:25:06.826 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQACG45pndKWKhAAqV+JKZ54KVP7kABF/obBrg== --osd-uuid af288f34-dfe7-42f0-8ea0-bc7e9610fa17 2026-02-12T18:25:06.843 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:06.839+0000 7f9ccda34780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:06.844 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:06.840+0000 7f9ccda34780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:06.845 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:06.841+0000 7f9ccda34780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:06.845 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:06.841+0000 7f9ccda34780 -1 bdev(0x285cc00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-12T18:25:06.845 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:06.841+0000 7f9ccda34780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-12T18:25:07.281 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-12T18:25:07.282 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:25:07.282 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T18:25:07.283 INFO:tasks.workunit.client.0.trial116.stdout:adding osd0 key to auth repository 2026-02-12T18:25:07.283 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:25:07.385 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T18:25:07.386 INFO:tasks.workunit.client.0.trial116.stdout:start osd.0 2026-02-12T18:25:07.386 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:25:07.386 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:25:07.386 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:25:07.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:25:07.400 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:07.396+0000 7f7c98262780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:07.402 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:07.398+0000 7f7c98262780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:07.402 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:07.399+0000 7f7c98262780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:07.485 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T18:25:07.485 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:25:07.485 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T18:25:07.485 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:25:07.485 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:25:07.485 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:25:07.485 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:25:07.486 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:25:07.486 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:25:07.486 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:25:07.584 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:25:07.604 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:07.601+0000 7f7c98262780 -1 Falling back to public interface 2026-02-12T18:25:07.850 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:07.846+0000 7f7c98262780 -1 osd.0 0 log_to_monitors true 2026-02-12T18:25:08.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:25:08.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:25:08.586 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:25:08.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:25:08.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:25:08.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:25:08.821 INFO:tasks.workunit.client.0.trial116.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1723176386,v1:127.0.0.1:6803/1723176386] [v2:127.0.0.1:6804/1723176386,v1:127.0.0.1:6805/1723176386] exists,up af288f34-dfe7-42f0-8ea0-bc7e9610fa17 2026-02-12T18:25:08.826 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:25:08.826 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:25:08.826 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:25:08.826 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:25:08.826 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-12T18:25:08.826 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:25:08.826 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:25:08.826 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-12T18:25:08.826 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:25:08.826 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-12T18:25:08.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:25:08.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:25:08.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:25:08.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:25:08.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-12T18:25:08.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-12T18:25:08.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:25:08.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:25:08.827 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:25:08.827 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:25:08.827 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:25:08.827 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:25:08.828 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:25:08.828 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:08.828 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:08.828 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:25:08.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:25:08.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:25:08.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:25:08.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:25:08.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:25:08.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:25:08.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:25:08.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:25:08.828 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:25:08.829 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:25:08.829 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:25:08.829 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:25:08.829 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-12T18:25:08.829 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:25:08.829 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1b6a428f-3c2d-49a4-bc1a-36503507cb8a 2026-02-12T18:25:08.829 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 1b6a428f-3c2d-49a4-bc1a-36503507cb8a' 2026-02-12T18:25:08.829 INFO:tasks.workunit.client.0.trial116.stdout:add osd1 1b6a428f-3c2d-49a4-bc1a-36503507cb8a 2026-02-12T18:25:08.830 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:25:08.835 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAEG45pe4GTMRAAedaNTGgkiHHEL0R1mUQdeQ== 2026-02-12T18:25:08.835 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAEG45pe4GTMRAAedaNTGgkiHHEL0R1mUQdeQ=="}' 2026-02-12T18:25:08.835 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1b6a428f-3c2d-49a4-bc1a-36503507cb8a -i td/test-erasure-eio/1/new.json 2026-02-12T18:25:09.070 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:25:09.079 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-12T18:25:09.080 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAEG45pe4GTMRAAedaNTGgkiHHEL0R1mUQdeQ== --osd-uuid 1b6a428f-3c2d-49a4-bc1a-36503507cb8a 2026-02-12T18:25:09.096 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:09.092+0000 7f51713d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:09.097 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:09.093+0000 7f51713d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:09.098 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:09.094+0000 7f51713d8780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:09.098 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:09.094+0000 7f51713d8780 -1 bdev(0x31bcc00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-12T18:25:09.098 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:09.094+0000 7f51713d8780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-12T18:25:09.422 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-12T18:25:09.422 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:25:09.423 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-12T18:25:09.423 INFO:tasks.workunit.client.0.trial116.stdout:adding osd1 key to auth repository 2026-02-12T18:25:09.423 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:25:09.744 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-12T18:25:09.745 INFO:tasks.workunit.client.0.trial116.stdout:start osd.1 2026-02-12T18:25:09.745 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:25:09.746 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:25:09.746 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:25:09.746 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:25:09.759 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:09.755+0000 7f7392dd4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:09.760 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:09.757+0000 7f7392dd4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:09.761 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:09.758+0000 7f7392dd4780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:09.895 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:09.891+0000 7f7392dd4780 -1 Falling back to public interface 2026-02-12T18:25:09.972 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-12T18:25:09.972 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:25:09.972 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T18:25:09.972 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:25:09.972 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:25:09.972 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:25:09.973 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:25:09.973 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:25:09.973 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:25:09.974 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:25:10.181 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:10.178+0000 7f7392dd4780 -1 osd.1 0 log_to_monitors true 2026-02-12T18:25:10.199 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:25:11.200 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:25:11.201 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:25:11.201 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:25:11.202 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:25:11.202 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:25:11.202 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:25:11.426 INFO:tasks.workunit.client.0.trial116.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/184136473,v1:127.0.0.1:6811/184136473] [v2:127.0.0.1:6812/184136473,v1:127.0.0.1:6813/184136473] exists,up 1b6a428f-3c2d-49a4-bc1a-36503507cb8a 2026-02-12T18:25:11.429 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:25:11.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:25:11.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:25:11.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:25:11.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-12T18:25:11.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:25:11.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:25:11.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-12T18:25:11.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:25:11.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-12T18:25:11.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:25:11.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:25:11.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:25:11.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:25:11.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-12T18:25:11.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-12T18:25:11.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:25:11.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:25:11.431 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:25:11.431 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:25:11.431 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:25:11.431 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:25:11.432 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:25:11.432 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:11.432 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:11.432 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:25:11.432 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:25:11.432 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:25:11.432 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:25:11.432 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:25:11.432 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:25:11.432 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:25:11.432 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:25:11.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:25:11.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:25:11.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:25:11.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:25:11.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:25:11.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-12T18:25:11.433 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:25:11.434 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5ccce218-91d4-4001-b346-984700ea731c 2026-02-12T18:25:11.434 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 5ccce218-91d4-4001-b346-984700ea731c' 2026-02-12T18:25:11.434 INFO:tasks.workunit.client.0.trial116.stdout:add osd2 5ccce218-91d4-4001-b346-984700ea731c 2026-02-12T18:25:11.435 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:25:11.440 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAHG45plCwLGhAAoyxMkbi2sEpecg1c/eVrtQ== 2026-02-12T18:25:11.440 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAHG45plCwLGhAAoyxMkbi2sEpecg1c/eVrtQ=="}' 2026-02-12T18:25:11.440 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5ccce218-91d4-4001-b346-984700ea731c -i td/test-erasure-eio/2/new.json 2026-02-12T18:25:11.671 INFO:tasks.workunit.client.0.trial116.stdout:2 2026-02-12T18:25:11.679 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-12T18:25:11.680 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAHG45plCwLGhAAoyxMkbi2sEpecg1c/eVrtQ== --osd-uuid 5ccce218-91d4-4001-b346-984700ea731c 2026-02-12T18:25:11.696 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:11.692+0000 7f0f9cc0e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:11.697 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:11.694+0000 7f0f9cc0e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:11.698 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:11.695+0000 7f0f9cc0e780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:11.699 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:11.695+0000 7f0f9cc0e780 -1 bdev(0x2d00c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-12T18:25:11.699 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:11.695+0000 7f0f9cc0e780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-12T18:25:12.021 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-12T18:25:12.021 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:25:12.022 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-12T18:25:12.022 INFO:tasks.workunit.client.0.trial116.stdout:adding osd2 key to auth repository 2026-02-12T18:25:12.022 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:25:12.351 INFO:tasks.workunit.client.0.trial116.stdout:start osd.2 2026-02-12T18:25:12.352 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-12T18:25:12.352 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:25:12.353 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:25:12.353 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:25:12.353 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:25:12.366 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:12.362+0000 7f3ebb833780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:12.367 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:12.364+0000 7f3ebb833780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:12.368 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:12.365+0000 7f3ebb833780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:12.508 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:12.504+0000 7f3ebb833780 -1 Falling back to public interface 2026-02-12T18:25:12.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-12T18:25:12.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:25:12.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T18:25:12.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:25:12.586 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:25:12.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:25:12.587 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:25:12.587 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:25:12.588 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:25:12.588 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:25:12.808 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:25:12.890 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:12.887+0000 7f3ebb833780 -1 osd.2 0 log_to_monitors true 2026-02-12T18:25:13.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:25:13.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:25:13.810 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:25:13.810 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:25:13.810 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:25:13.811 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:25:14.050 INFO:tasks.workunit.client.0.trial116.stdout:osd.2 up in weight 1 up_from 18 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3895516034,v1:127.0.0.1:6819/3895516034] [v2:127.0.0.1:6820/3895516034,v1:127.0.0.1:6821/3895516034] exists,up 5ccce218-91d4-4001-b346-984700ea731c 2026-02-12T18:25:14.054 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:25:14.055 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:25:14.055 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:25:14.055 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:25:14.055 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-12T18:25:14.055 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:25:14.055 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:25:14.055 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-12T18:25:14.055 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:25:14.055 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-12T18:25:14.056 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:25:14.056 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:25:14.056 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:25:14.056 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:25:14.056 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-12T18:25:14.056 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-12T18:25:14.056 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:25:14.056 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:25:14.056 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:25:14.056 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:25:14.056 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:25:14.056 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:25:14.057 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:25:14.057 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:14.057 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:14.057 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:25:14.057 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:25:14.057 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:25:14.057 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:25:14.057 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:25:14.057 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:25:14.057 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:25:14.057 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:25:14.057 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:25:14.057 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:25:14.058 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:25:14.058 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:25:14.058 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:25:14.058 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-12T18:25:14.058 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:25:14.058 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a1ecfd6e-c9e7-4d19-9f84-3792a06026da 2026-02-12T18:25:14.058 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 a1ecfd6e-c9e7-4d19-9f84-3792a06026da' 2026-02-12T18:25:14.058 INFO:tasks.workunit.client.0.trial116.stdout:add osd3 a1ecfd6e-c9e7-4d19-9f84-3792a06026da 2026-02-12T18:25:14.059 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:25:14.065 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAKG45pI7arAxAAESs76Vj4ADmJJwFdaZ0rPg== 2026-02-12T18:25:14.065 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAKG45pI7arAxAAESs76Vj4ADmJJwFdaZ0rPg=="}' 2026-02-12T18:25:14.065 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a1ecfd6e-c9e7-4d19-9f84-3792a06026da -i td/test-erasure-eio/3/new.json 2026-02-12T18:25:14.293 INFO:tasks.workunit.client.0.trial116.stdout:3 2026-02-12T18:25:14.300 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-12T18:25:14.302 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAKG45pI7arAxAAESs76Vj4ADmJJwFdaZ0rPg== --osd-uuid a1ecfd6e-c9e7-4d19-9f84-3792a06026da 2026-02-12T18:25:14.317 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:14.313+0000 7f1b3703b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:14.319 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:14.315+0000 7f1b3703b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:14.319 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:14.316+0000 7f1b3703b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:14.320 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:14.316+0000 7f1b3703b780 -1 bdev(0x241ec00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-12T18:25:14.320 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:14.316+0000 7f1b3703b780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-12T18:25:14.642 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-12T18:25:14.642 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:25:14.643 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-12T18:25:14.643 INFO:tasks.workunit.client.0.trial116.stdout:adding osd3 key to auth repository 2026-02-12T18:25:14.643 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:25:14.961 INFO:tasks.workunit.client.0.trial116.stdout:start osd.3 2026-02-12T18:25:14.962 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-12T18:25:14.963 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:25:14.963 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:25:14.963 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:25:14.963 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:25:14.976 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:14.972+0000 7f90cdc37780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:14.977 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:14.974+0000 7f90cdc37780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:14.978 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:14.975+0000 7f90cdc37780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:15.115 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:15.111+0000 7f90cdc37780 -1 Falling back to public interface 2026-02-12T18:25:15.188 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-12T18:25:15.188 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:25:15.188 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-12T18:25:15.188 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:25:15.188 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:25:15.188 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:25:15.188 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:25:15.189 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:25:15.189 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:25:15.189 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:25:15.417 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:25:15.437 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:15.433+0000 7f90cdc37780 -1 osd.3 0 log_to_monitors true 2026-02-12T18:25:16.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:25:16.419 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:25:16.419 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:25:16.420 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:25:16.420 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:25:16.420 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:25:16.657 INFO:tasks.workunit.client.0.trial116.stdout:osd.3 up in weight 1 up_from 26 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2397144694,v1:127.0.0.1:6827/2397144694] [v2:127.0.0.1:6828/2397144694,v1:127.0.0.1:6829/2397144694] exists,up a1ecfd6e-c9e7-4d19-9f84-3792a06026da 2026-02-12T18:25:16.658 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:25:16.658 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:25:16.658 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:25:16.658 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-12T18:25:16.658 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T18:25:16.658 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T18:25:16.659 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:25:16.659 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:16.659 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:16.659 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.0.asok 2026-02-12T18:25:16.659 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-12T18:25:16.659 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-osd.0.asok log flush 2026-02-12T18:25:16.704 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:25:16.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-12T18:25:16.710 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:25:07.603+0000 7f7c98262780 0 load: jerasure load: lrc 2026-02-12T18:25:16.711 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:305: TEST_rados_get_subread_missing: local poolname=pool-jerasure 2026-02-12T18:25:16.712 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:306: TEST_rados_get_subread_missing: create_erasure_coded_pool pool-jerasure 2 1 2026-02-12T18:25:16.712 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-12T18:25:16.712 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-12T18:25:16.712 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-12T18:25:16.712 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-12T18:25:16.712 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-12T18:25:16.712 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-12T18:25:16.712 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-12T18:25:17.039 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-12T18:25:17.040 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-12T18:25:17.413 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-jerasure' already exists 2026-02-12T18:25:17.421 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:25:18.423 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-12T18:25:18.423 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:25:18.423 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:25:18.424 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:25:18.425 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:25:18.425 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:25:18.425 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:25:18.425 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:25:18.425 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:25:18.425 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:25:18.514 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:25:18.514 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:25:18.514 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:25:18.514 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:25:18.514 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:25:18.514 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:25:18.737 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:25:18.737 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:25:18.737 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:25:18.737 INFO:tasks.workunit.client.0.trial116.stderr:3' 2026-02-12T18:25:18.737 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:25:18.737 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:25:18.737 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:25:18.803 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803780 2026-02-12T18:25:18.803 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803780 2026-02-12T18:25:18.803 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780' 2026-02-12T18:25:18.803 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:25:18.804 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:25:18.868 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574852 2026-02-12T18:25:18.868 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574852 2026-02-12T18:25:18.868 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-55834574852' 2026-02-12T18:25:18.869 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:25:18.869 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:25:18.931 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411332 2026-02-12T18:25:18.932 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411332 2026-02-12T18:25:18.932 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-55834574852 2-77309411332' 2026-02-12T18:25:18.932 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:25:18.932 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:25:18.998 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149699 2026-02-12T18:25:18.998 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149699 2026-02-12T18:25:18.999 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-55834574852 2-77309411332 3-111669149699' 2026-02-12T18:25:18.999 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:25:18.999 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803780 2026-02-12T18:25:18.999 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:25:18.999 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:25:19.000 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803780 2026-02-12T18:25:19.000 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:25:19.001 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803780 2026-02-12T18:25:19.001 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803780' 2026-02-12T18:25:19.002 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.0 seq 25769803780 2026-02-12T18:25:19.002 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:25:19.227 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803778 -lt 25769803780 2026-02-12T18:25:19.227 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:25:20.228 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:25:20.229 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:25:20.454 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803780 2026-02-12T18:25:20.454 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:25:20.454 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-55834574852 2026-02-12T18:25:20.454 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:25:20.455 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:25:20.456 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574852 2026-02-12T18:25:20.456 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:25:20.457 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574852 2026-02-12T18:25:20.457 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 55834574852' 2026-02-12T18:25:20.458 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.1 seq 55834574852 2026-02-12T18:25:20.458 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:25:20.684 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574852 -lt 55834574852 2026-02-12T18:25:20.684 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:25:20.684 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411332 2026-02-12T18:25:20.685 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:25:20.685 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:25:20.686 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411332 2026-02-12T18:25:20.686 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:25:20.687 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411332 2026-02-12T18:25:20.687 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411332' 2026-02-12T18:25:20.688 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.2 seq 77309411332 2026-02-12T18:25:20.688 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:25:20.924 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411333 -lt 77309411332 2026-02-12T18:25:20.925 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:25:20.925 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149699 2026-02-12T18:25:20.925 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:25:20.926 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:25:20.927 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149699 2026-02-12T18:25:20.927 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:25:20.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149699 2026-02-12T18:25:20.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149699' 2026-02-12T18:25:20.928 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.3 seq 111669149699 2026-02-12T18:25:20.929 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:25:21.158 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149699 -lt 111669149699 2026-02-12T18:25:21.158 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:25:21.159 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:25:21.159 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:25:21.482 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:25:21.482 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:25:21.483 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:25:21.483 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:25:21.483 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:25:21.483 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:25:21.483 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:25:21.483 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:25:21.707 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:25:21.708 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:25:21.708 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:25:21.708 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:25:22.026 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:25:22.026 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:25:22.026 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:25:22.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:308: TEST_rados_get_subread_missing: local shard_id=1 2026-02-12T18:25:22.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:309: TEST_rados_get_subread_missing: rados_put_get_data remove td/test-erasure-eio 1 2026-02-12T18:25:22.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=remove 2026-02-12T18:25:22.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-12T18:25:22.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-12T18:25:22.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-12T18:25:22.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=1 2026-02-12T18:25:22.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-12T18:25:22.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg= 2026-02-12T18:25:22.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-12T18:25:22.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-remove-76821-1 2026-02-12T18:25:22.028 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-remove-76821-1 2026-02-12T18:25:22.028 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-12T18:25:22.028 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-12T18:25:22.028 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-remove-76821-1 2026-02-12T18:25:22.028 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:25:22.028 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-12T18:25:22.028 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:25:22.028 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-12T18:25:22.028 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:25:22.028 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-12T18:25:22.028 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:25:22.028 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-12T18:25:22.028 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-remove-76821-1 td/test-erasure-eio/ORIGINAL 2026-02-12T18:25:22.043 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_remove ec data pool-jerasure obj-remove-76821-1 td/test-erasure-eio 1 2026-02-12T18:25:22.043 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:127: inject_remove: local pooltype=ec 2026-02-12T18:25:22.043 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:128: inject_remove: shift 2026-02-12T18:25:22.043 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:129: inject_remove: local which=data 2026-02-12T18:25:22.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:130: inject_remove: shift 2026-02-12T18:25:22.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:131: inject_remove: local poolname=pool-jerasure 2026-02-12T18:25:22.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:132: inject_remove: shift 2026-02-12T18:25:22.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:133: inject_remove: local objname=obj-remove-76821-1 2026-02-12T18:25:22.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:134: inject_remove: shift 2026-02-12T18:25:22.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:135: inject_remove: local dir=td/test-erasure-eio 2026-02-12T18:25:22.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:136: inject_remove: shift 2026-02-12T18:25:22.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:137: inject_remove: local shard_id=1 2026-02-12T18:25:22.045 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:138: inject_remove: shift 2026-02-12T18:25:22.045 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: get_osds pool-jerasure obj-remove-76821-1 2026-02-12T18:25:22.045 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:25:22.045 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-remove-76821-1 2026-02-12T18:25:22.045 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-remove-76821-1 2026-02-12T18:25:22.045 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:25:22.275 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:25:22.275 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:25:22.275 INFO:tasks.workunit.client.0.trial116.stderr:2' 2026-02-12T18:25:22.275 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-12T18:25:22.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: initial_osds=('3' '1' '2') 2026-02-12T18:25:22.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: local -a initial_osds 2026-02-12T18:25:22.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:141: inject_remove: local osd_id=1 2026-02-12T18:25:22.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:142: inject_remove: objectstore_tool td/test-erasure-eio 1 obj-remove-76821-1 remove 2026-02-12T18:25:22.276 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-12T18:25:22.276 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-12T18:25:22.276 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-12T18:25:22.276 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-12T18:25:22.276 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-remove-76821-1 remove 2026-02-12T18:25:22.276 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-12T18:25:22.276 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-12T18:25:22.276 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-12T18:25:22.276 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-12T18:25:22.276 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-12T18:25:22.276 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:25:22.276 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:25:22.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:25:22.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:25:22.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:25:22.581 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:25:22.582 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-remove-76821-1 remove 2026-02-12T18:25:22.582 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-12T18:25:22.582 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-12T18:25:22.582 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-12T18:25:22.582 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-12T18:25:22.582 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-12T18:25:22.582 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-remove-76821-1 remove 2026-02-12T18:25:22.690 INFO:tasks.workunit.client.0.trial116.stdout:remove 1#2:90291ca8:::obj-remove-76821-1:head# 2026-02-12T18:25:22.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-12T18:25:22.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-12T18:25:22.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T18:25:22.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-12T18:25:22.731 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T18:25:22.732 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-12T18:25:22.732 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:25:22.732 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:25:22.732 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:25:22.732 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:25:22.732 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-12T18:25:22.732 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-12T18:25:22.732 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T18:25:22.732 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T18:25:22.732 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:25:22.732 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T18:25:22.733 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:25:22.733 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:25:22.733 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:25:22.733 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:22.733 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:22.733 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:25:22.733 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:25:22.733 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:25:22.733 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:25:22.733 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:25:22.733 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:25:22.733 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:25:22.733 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:25:22.734 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:25:22.734 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T18:25:22.734 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-12T18:25:22.734 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-12T18:25:22.734 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-12T18:25:22.734 INFO:tasks.workunit.client.0.trial116.stderr:start osd.1 2026-02-12T18:25:22.734 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:25:22.735 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-12T18:25:22.735 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-12T18:25:22.735 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T18:25:22.735 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T18:25:22.735 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T18:25:22.746 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:22.743+0000 7fb997643780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:22.747 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:22.744+0000 7fb997643780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:22.748 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:22.745+0000 7fb997643780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:22.869 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:22.867+0000 7fb997643780 -1 Falling back to public interface 2026-02-12T18:25:22.966 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-12T18:25:22.966 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:25:22.967 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T18:25:22.967 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:25:22.967 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:25:22.967 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:25:22.967 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:25:22.967 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:25:22.967 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:25:22.967 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:25:23.179 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:23.176+0000 7fb997643780 -1 osd.1 35 log_to_monitors true 2026-02-12T18:25:23.196 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:25:24.198 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:25:24.198 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:25:24.198 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:25:24.198 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:25:24.199 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:25:24.199 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:25:24.422 INFO:tasks.workunit.client.0.trial116.stderr:osd.1 up in weight 1 up_from 39 up_thru 39 down_at 36 last_clean_interval [13,35) [v2:127.0.0.1:6810/3927352418,v1:127.0.0.1:6811/3927352418] [v2:127.0.0.1:6812/3927352418,v1:127.0.0.1:6813/3927352418] exists,up 1b6a428f-3c2d-49a4-bc1a-36503507cb8a 2026-02-12T18:25:24.423 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:25:24.423 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:25:24.423 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:25:24.423 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-12T18:25:24.423 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:25:24.423 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:25:24.423 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:25:24.423 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:25:24.424 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:25:24.424 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:25:24.424 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:25:24.424 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:25:24.424 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:25:24.510 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:25:24.510 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:25:24.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:25:24.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:25:24.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:25:24.511 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:25:24.735 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:25:24.736 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:25:24.736 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:25:24.736 INFO:tasks.workunit.client.0.trial116.stderr:3' 2026-02-12T18:25:24.736 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:25:24.736 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:25:24.736 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:25:24.800 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803784 2026-02-12T18:25:24.800 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803784 2026-02-12T18:25:24.800 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784' 2026-02-12T18:25:24.800 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:25:24.800 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:25:24.863 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724547 2026-02-12T18:25:24.863 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724547 2026-02-12T18:25:24.863 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-167503724547' 2026-02-12T18:25:24.863 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:25:24.864 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:25:24.927 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411336 2026-02-12T18:25:24.927 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411336 2026-02-12T18:25:24.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-167503724547 2-77309411336' 2026-02-12T18:25:24.928 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:25:24.928 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:25:24.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149702 2026-02-12T18:25:24.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149702 2026-02-12T18:25:24.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-167503724547 2-77309411336 3-111669149702' 2026-02-12T18:25:24.993 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:25:24.994 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803784 2026-02-12T18:25:24.994 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:25:24.994 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:25:24.995 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803784 2026-02-12T18:25:24.995 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:25:24.996 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803784 2026-02-12T18:25:24.996 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-12T18:25:24.996 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.0 seq 25769803784 2026-02-12T18:25:24.997 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:25:25.222 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803784 2026-02-12T18:25:25.222 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:25:26.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:25:26.224 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:25:26.447 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-12T18:25:26.447 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:25:26.448 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-167503724547 2026-02-12T18:25:26.448 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:25:26.449 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:25:26.449 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-167503724547 2026-02-12T18:25:26.449 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:25:26.451 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=167503724547 2026-02-12T18:25:26.451 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 167503724547' 2026-02-12T18:25:26.451 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.1 seq 167503724547 2026-02-12T18:25:26.451 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:25:26.675 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 167503724547 -lt 167503724547 2026-02-12T18:25:26.675 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:25:26.676 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411336 2026-02-12T18:25:26.676 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:25:26.677 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:25:26.677 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411336 2026-02-12T18:25:26.677 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:25:26.678 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411336 2026-02-12T18:25:26.679 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411336' 2026-02-12T18:25:26.679 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.2 seq 77309411336 2026-02-12T18:25:26.679 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:25:26.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411336 -lt 77309411336 2026-02-12T18:25:26.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:25:26.904 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149702 2026-02-12T18:25:26.905 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:25:26.905 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:25:26.906 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149702 2026-02-12T18:25:26.906 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:25:26.907 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149702 2026-02-12T18:25:26.907 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149702' 2026-02-12T18:25:26.907 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.3 seq 111669149702 2026-02-12T18:25:26.908 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:25:27.133 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149702 -lt 111669149702 2026-02-12T18:25:27.133 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:25:27.134 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:25:27.134 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:25:27.447 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:25:27.447 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:25:27.447 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:25:27.447 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:25:27.447 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:25:27.447 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:25:27.447 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:25:27.447 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:25:27.671 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:25:27.671 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:25:27.672 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:25:27.672 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:25:27.987 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:25:27.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:25:27.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:25:27.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-remove-76821-1 2026-02-12T18:25:27.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-12T18:25:27.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-12T18:25:27.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-remove-76821-1 2026-02-12T18:25:27.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-12T18:25:27.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-12T18:25:27.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-remove-76821-1 td/test-erasure-eio/COPY 2026-02-12T18:25:28.005 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-12T18:25:28.005 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-12T18:25:28.006 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' '' = recovery ']' 2026-02-12T18:25:28.006 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: expr 1 + 1 2026-02-12T18:25:28.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: local another_shard_id=2 2026-02-12T18:25:28.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:182: rados_put_get_data: inject_remove ec data pool-jerasure obj-remove-76821-1 td/test-erasure-eio 2 2026-02-12T18:25:28.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:127: inject_remove: local pooltype=ec 2026-02-12T18:25:28.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:128: inject_remove: shift 2026-02-12T18:25:28.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:129: inject_remove: local which=data 2026-02-12T18:25:28.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:130: inject_remove: shift 2026-02-12T18:25:28.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:131: inject_remove: local poolname=pool-jerasure 2026-02-12T18:25:28.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:132: inject_remove: shift 2026-02-12T18:25:28.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:133: inject_remove: local objname=obj-remove-76821-1 2026-02-12T18:25:28.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:134: inject_remove: shift 2026-02-12T18:25:28.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:135: inject_remove: local dir=td/test-erasure-eio 2026-02-12T18:25:28.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:136: inject_remove: shift 2026-02-12T18:25:28.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:137: inject_remove: local shard_id=2 2026-02-12T18:25:28.010 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:138: inject_remove: shift 2026-02-12T18:25:28.010 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: get_osds pool-jerasure obj-remove-76821-1 2026-02-12T18:25:28.010 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:25:28.010 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-remove-76821-1 2026-02-12T18:25:28.010 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-remove-76821-1 2026-02-12T18:25:28.011 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:25:28.242 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:25:28.242 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:25:28.242 INFO:tasks.workunit.client.0.trial116.stderr:2' 2026-02-12T18:25:28.242 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-12T18:25:28.243 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: initial_osds=('3' '1' '2') 2026-02-12T18:25:28.243 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: local -a initial_osds 2026-02-12T18:25:28.243 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:141: inject_remove: local osd_id=2 2026-02-12T18:25:28.243 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:142: inject_remove: objectstore_tool td/test-erasure-eio 2 obj-remove-76821-1 remove 2026-02-12T18:25:28.243 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-12T18:25:28.243 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-12T18:25:28.243 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-12T18:25:28.243 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-12T18:25:28.243 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-remove-76821-1 remove 2026-02-12T18:25:28.243 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-12T18:25:28.243 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-12T18:25:28.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-12T18:25:28.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-12T18:25:28.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-12T18:25:28.244 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:25:28.244 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:25:28.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:25:28.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:25:28.244 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:25:28.549 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:25:28.549 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-remove-76821-1 remove 2026-02-12T18:25:28.549 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-12T18:25:28.549 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-12T18:25:28.549 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-12T18:25:28.549 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-12T18:25:28.549 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-12T18:25:28.549 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-remove-76821-1 remove 2026-02-12T18:25:28.656 INFO:tasks.workunit.client.0.trial116.stdout:remove 2#2:90291ca8:::obj-remove-76821-1:head# 2026-02-12T18:25:28.703 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-12T18:25:28.703 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-12T18:25:28.703 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T18:25:28.703 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-12T18:25:28.703 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T18:25:28.703 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-12T18:25:28.704 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:25:28.704 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:25:28.704 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:25:28.704 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:25:28.704 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-12T18:25:28.704 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-12T18:25:28.704 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T18:25:28.704 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T18:25:28.704 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:25:28.704 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T18:25:28.704 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:25:28.704 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:25:28.705 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:25:28.705 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:28.705 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:28.705 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:25:28.705 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:25:28.705 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:25:28.705 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:25:28.705 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:25:28.705 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:25:28.705 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:25:28.705 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:25:28.705 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:25:28.705 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T18:25:28.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-12T18:25:28.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-12T18:25:28.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-12T18:25:28.706 INFO:tasks.workunit.client.0.trial116.stderr:start osd.2 2026-02-12T18:25:28.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:25:28.706 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-12T18:25:28.707 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-12T18:25:28.707 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T18:25:28.707 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T18:25:28.707 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T18:25:28.717 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:28.714+0000 7f2f5f855780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:28.719 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:28.716+0000 7f2f5f855780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:28.720 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:28.717+0000 7f2f5f855780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:28.845 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:28.842+0000 7f2f5f855780 -1 Falling back to public interface 2026-02-12T18:25:28.929 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-12T18:25:28.929 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:25:28.929 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T18:25:28.929 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:25:28.929 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:25:28.930 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:25:28.930 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:25:28.930 INFO:tasks.workunit.client.0.trial116.stderr:0 2026-02-12T18:25:28.930 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:25:28.930 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:25:29.146 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:29.143+0000 7f2f5f855780 -1 osd.2 42 log_to_monitors true 2026-02-12T18:25:29.157 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:25:30.159 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:25:30.159 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:25:30.159 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:25:30.160 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:25:30.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:25:30.160 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:25:30.387 INFO:tasks.workunit.client.0.trial116.stderr:osd.2 up in weight 1 up_from 45 up_thru 36 down_at 42 last_clean_interval [18,42) [v2:127.0.0.1:6818/2888494394,v1:127.0.0.1:6819/2888494394] [v2:127.0.0.1:6820/2888494394,v1:127.0.0.1:6821/2888494394] exists,up 5ccce218-91d4-4001-b346-984700ea731c 2026-02-12T18:25:30.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:25:30.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:25:30.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:25:30.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-12T18:25:30.387 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:25:30.388 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:25:30.388 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:25:30.388 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:25:30.388 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:25:30.389 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:25:30.389 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:25:30.389 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:25:30.389 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:25:30.475 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:25:30.476 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:25:30.476 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:25:30.476 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:25:30.476 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:25:30.476 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:25:30.701 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:25:30.701 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:25:30.701 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:25:30.701 INFO:tasks.workunit.client.0.trial116.stderr:3' 2026-02-12T18:25:30.701 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:25:30.701 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:25:30.701 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:25:30.769 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803787 2026-02-12T18:25:30.770 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803787 2026-02-12T18:25:30.770 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787' 2026-02-12T18:25:30.770 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:25:30.770 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:25:30.835 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724550 2026-02-12T18:25:30.835 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724550 2026-02-12T18:25:30.835 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-167503724550' 2026-02-12T18:25:30.835 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:25:30.835 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:25:30.900 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=193273528323 2026-02-12T18:25:30.900 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 193273528323 2026-02-12T18:25:30.900 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-167503724550 2-193273528323' 2026-02-12T18:25:30.900 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:25:30.901 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:25:30.963 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149705 2026-02-12T18:25:30.963 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149705 2026-02-12T18:25:30.964 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803787 1-167503724550 2-193273528323 3-111669149705' 2026-02-12T18:25:30.964 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:25:30.964 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803787 2026-02-12T18:25:30.964 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:25:30.964 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:25:30.965 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803787 2026-02-12T18:25:30.965 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:25:30.966 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803787 2026-02-12T18:25:30.966 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803787' 2026-02-12T18:25:30.966 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.0 seq 25769803787 2026-02-12T18:25:30.967 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:25:31.194 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803785 -lt 25769803787 2026-02-12T18:25:31.194 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:25:32.195 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:25:32.196 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:25:32.422 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803787 -lt 25769803787 2026-02-12T18:25:32.422 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:25:32.423 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-167503724550 2026-02-12T18:25:32.423 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:25:32.424 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:25:32.425 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-167503724550 2026-02-12T18:25:32.425 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:25:32.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=167503724550 2026-02-12T18:25:32.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 167503724550' 2026-02-12T18:25:32.426 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.1 seq 167503724550 2026-02-12T18:25:32.427 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:25:32.650 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 167503724550 -lt 167503724550 2026-02-12T18:25:32.650 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:25:32.650 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-193273528323 2026-02-12T18:25:32.650 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:25:32.651 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:25:32.652 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-193273528323 2026-02-12T18:25:32.652 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:25:32.653 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=193273528323 2026-02-12T18:25:32.653 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 193273528323' 2026-02-12T18:25:32.654 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.2 seq 193273528323 2026-02-12T18:25:32.654 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:25:32.878 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 193273528323 -lt 193273528323 2026-02-12T18:25:32.878 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:25:32.878 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149705 2026-02-12T18:25:32.879 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:25:32.879 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:25:32.880 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149705 2026-02-12T18:25:32.880 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:25:32.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149705 2026-02-12T18:25:32.881 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149705' 2026-02-12T18:25:32.881 INFO:tasks.workunit.client.0.trial116.stderr:waiting osd.3 seq 111669149705 2026-02-12T18:25:32.882 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:25:33.110 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149705 -lt 111669149705 2026-02-12T18:25:33.111 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:25:33.111 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:25:33.111 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:25:33.423 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:25:33.424 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:25:33.425 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:25:33.425 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:25:33.425 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:25:33.425 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:25:33.425 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:25:33.425 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:25:33.647 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:25:33.648 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:25:33.648 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:25:33.648 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:25:33.985 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:25:33.985 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:25:33.985 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:25:33.986 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:183: rados_put_get_data: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-12T18:25:33.986 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:188: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-remove-76821-1 2026-02-12T18:25:33.986 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-12T18:25:33.986 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-12T18:25:33.986 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-remove-76821-1 2026-02-12T18:25:33.986 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-12T18:25:33.986 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-12T18:25:33.986 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-remove-76821-1 td/test-erasure-eio/COPY 2026-02-12T18:25:34.001 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-12T18:25:34.002 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-12T18:25:34.003 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: rados_put_get_data: rm td/test-erasure-eio/ORIGINAL 2026-02-12T18:25:34.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:310: TEST_rados_get_subread_missing: delete_erasure_coded_pool pool-jerasure 2026-02-12T18:25:34.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-12T18:25:34.004 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-12T18:25:34.275 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-jerasure' does not exist 2026-02-12T18:25:34.282 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-12T18:25:34.558 INFO:tasks.workunit.client.0.trial116.stderr:erasure-code-profile myprofile does not exist 2026-02-12T18:25:34.566 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-12T18:25:34.566 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-12T18:25:34.566 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:25:34.566 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-12T18:25:34.567 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:25:34.567 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:25:34.567 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:25:34.567 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:25:34.567 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:25:34.675 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:25:34.676 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:25:34.677 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:25:34.677 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:25:34.679 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:25:34.679 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:25:34.679 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:25:34.680 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:25:34.680 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:25:34.681 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:25:34.681 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:25:34.682 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:25:34.682 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:25:34.683 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-12T18:25:34.693 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:25:34.693 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:34.693 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:34.694 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76821 2026-02-12T18:25:34.695 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:25:34.695 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:25:34.695 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-12T18:25:34.696 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-12T18:25:34.696 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-12T18:25:34.696 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-12T18:25:34.696 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-12T18:25:34.696 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:25:34.696 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-12T18:25:34.696 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:25:34.696 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:25:34.696 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:25:34.696 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:25:34.696 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:25:34.697 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:25:34.697 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:25:34.698 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:25:34.698 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:25:34.699 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:25:34.699 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:25:34.699 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:25:34.700 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:25:34.700 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:25:34.701 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:25:34.701 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:25:34.702 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:25:34.703 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:25:34.703 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-12T18:25:34.704 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:25:34.704 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:34.704 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:34.704 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76821 2026-02-12T18:25:34.705 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:25:34.705 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:25:34.705 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-12T18:25:34.706 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T18:25:34.706 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:34.706 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:34.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76821 2026-02-12T18:25:34.707 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T18:25:34.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T18:25:34.709 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T18:25:34.709 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-12T18:25:34.709 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-12T18:25:34.709 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-12T18:25:34.709 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T18:25:34.709 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T18:25:34.709 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T18:25:34.709 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-12T18:25:34.709 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-12T18:25:34.726 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T18:25:34.726 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:25:34.726 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:25:34.726 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:25:34.726 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:34.726 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:34.726 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:25:34.727 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T18:25:34.750 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T18:25:34.751 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T18:25:34.751 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:25:34.751 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:25:34.751 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T18:25:34.751 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T18:25:34.752 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:25:34.753 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:25:34.753 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:25:34.753 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:25:34.753 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:34.753 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:34.753 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:25:34.753 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:25:34.753 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76821/ceph-mon.a.asok config get fsid 2026-02-12T18:25:34.797 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T18:25:34.797 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:25:34.797 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:25:34.797 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T18:25:34.797 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T18:25:34.798 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:25:34.798 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:25:34.798 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:25:34.799 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:25:34.799 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:34.799 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:34.799 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:25:34.799 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:25:34.799 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76821/ceph-mon.a.asok config get mon_host 2026-02-12T18:25:34.840 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-12T18:25:34.840 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-12T18:25:34.840 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T18:25:34.840 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T18:25:34.841 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T18:25:34.841 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-12T18:25:34.841 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T18:25:34.942 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T18:25:34.942 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:25:34.942 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:25:34.942 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:25:34.942 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:34.943 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:34.943 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:25:34.943 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:25:34.943 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:25:34.958 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-12T18:25:34.959 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T18:25:35.053 INFO:tasks.workunit.client.0.trial116.stderr:pool 'rbd' already exists 2026-02-12T18:25:35.059 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:25:36.061 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-12T18:25:36.061 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:25:36.061 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:25:36.061 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:25:36.061 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:36.062 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:36.062 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:25:36.062 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-12T18:25:36.062 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-mon.a.asok log flush 2026-02-12T18:25:36.101 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:25:36.105 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-12T18:25:36.106 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:25:34.740+0000 7f676345ec00 0 load: jerasure load: lrc 2026-02-12T18:25:36.107 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_with_subreadall_eio_shard_0 td/test-erasure-eio 2026-02-12T18:25:36.107 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:350: TEST_rados_get_with_subreadall_eio_shard_0: local dir=td/test-erasure-eio 2026-02-12T18:25:36.107 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:351: TEST_rados_get_with_subreadall_eio_shard_0: local shard_id=0 2026-02-12T18:25:36.107 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:353: TEST_rados_get_with_subreadall_eio_shard_0: setup_osds 4 2026-02-12T18:25:36.108 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-12T18:25:36.108 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-12T18:25:36.108 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-12T18:25:36.108 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-12T18:25:36.112 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:25:36.112 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-12T18:25:36.112 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:25:36.112 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:25:36.112 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T18:25:36.112 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:25:36.113 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-12T18:25:36.113 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:25:36.113 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:25:36.113 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:25:36.113 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:25:36.113 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-12T18:25:36.113 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-12T18:25:36.113 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:25:36.113 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:25:36.113 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:25:36.113 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:25:36.113 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:25:36.114 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:25:36.114 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:25:36.114 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:36.114 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:36.114 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:25:36.114 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:25:36.114 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:25:36.114 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:25:36.114 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:25:36.114 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:25:36.114 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:25:36.114 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:25:36.114 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:25:36.115 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:25:36.115 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:25:36.115 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:25:36.115 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:25:36.115 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-12T18:25:36.115 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:25:36.115 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=52b14033-7666-4f2f-a636-4cc4aff2a872 2026-02-12T18:25:36.115 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 52b14033-7666-4f2f-a636-4cc4aff2a872' 2026-02-12T18:25:36.115 INFO:tasks.workunit.client.0.trial116.stdout:add osd0 52b14033-7666-4f2f-a636-4cc4aff2a872 2026-02-12T18:25:36.116 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:25:36.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAgG45pB6gWBxAAu2sRWZ8f5az6RrnOOPPg4g== 2026-02-12T18:25:36.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAgG45pB6gWBxAAu2sRWZ8f5az6RrnOOPPg4g=="}' 2026-02-12T18:25:36.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 52b14033-7666-4f2f-a636-4cc4aff2a872 -i td/test-erasure-eio/0/new.json 2026-02-12T18:25:36.225 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:25:36.231 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-12T18:25:36.232 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAgG45pB6gWBxAAu2sRWZ8f5az6RrnOOPPg4g== --osd-uuid 52b14033-7666-4f2f-a636-4cc4aff2a872 2026-02-12T18:25:36.248 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:36.244+0000 7f6352e51780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:36.249 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:36.245+0000 7f6352e51780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:36.250 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:36.246+0000 7f6352e51780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:36.250 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:36.246+0000 7f6352e51780 -1 bdev(0x31a2c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-12T18:25:36.250 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:36.246+0000 7f6352e51780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-12T18:25:36.689 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-12T18:25:36.689 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:25:36.690 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T18:25:36.690 INFO:tasks.workunit.client.0.trial116.stdout:adding osd0 key to auth repository 2026-02-12T18:25:36.690 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:25:36.792 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T18:25:36.792 INFO:tasks.workunit.client.0.trial116.stdout:start osd.0 2026-02-12T18:25:36.793 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:25:36.793 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:25:36.793 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:25:36.793 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:25:36.808 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:36.804+0000 7fb4f7c29780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:36.810 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:36.806+0000 7fb4f7c29780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:36.810 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:36.807+0000 7fb4f7c29780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:36.888 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T18:25:36.888 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:25:36.888 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T18:25:36.888 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:25:36.888 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:25:36.888 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:25:36.889 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:25:36.889 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:25:36.889 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:25:36.889 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:25:36.985 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:25:37.013 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:37.009+0000 7fb4f7c29780 -1 Falling back to public interface 2026-02-12T18:25:37.295 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:37.291+0000 7fb4f7c29780 -1 osd.0 0 log_to_monitors true 2026-02-12T18:25:37.987 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:25:37.987 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:25:37.987 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:25:37.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:25:37.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:25:37.988 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:25:38.217 INFO:tasks.workunit.client.0.trial116.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3123141676,v1:127.0.0.1:6803/3123141676] [v2:127.0.0.1:6804/3123141676,v1:127.0.0.1:6805/3123141676] exists,up 52b14033-7666-4f2f-a636-4cc4aff2a872 2026-02-12T18:25:38.221 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:25:38.221 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:25:38.221 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:25:38.221 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:25:38.221 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-12T18:25:38.221 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:25:38.221 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:25:38.221 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-12T18:25:38.221 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:25:38.222 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-12T18:25:38.222 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:25:38.222 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:25:38.222 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:25:38.222 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:25:38.222 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-12T18:25:38.222 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-12T18:25:38.222 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:25:38.222 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:25:38.222 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:25:38.222 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:25:38.222 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:25:38.222 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:25:38.223 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:25:38.223 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:38.223 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:38.223 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:25:38.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:25:38.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:25:38.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:25:38.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:25:38.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:25:38.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:25:38.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:25:38.223 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:25:38.224 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:25:38.224 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:25:38.224 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:25:38.224 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:25:38.224 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-12T18:25:38.224 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:25:38.224 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e9891a71-9e8e-45b8-8a94-81ca85f3d7ec 2026-02-12T18:25:38.224 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 e9891a71-9e8e-45b8-8a94-81ca85f3d7ec' 2026-02-12T18:25:38.224 INFO:tasks.workunit.client.0.trial116.stdout:add osd1 e9891a71-9e8e-45b8-8a94-81ca85f3d7ec 2026-02-12T18:25:38.225 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:25:38.231 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAiG45p/uiRDRAAcgwVlbC1WPTFnAMKFLLsdQ== 2026-02-12T18:25:38.231 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAiG45p/uiRDRAAcgwVlbC1WPTFnAMKFLLsdQ=="}' 2026-02-12T18:25:38.231 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e9891a71-9e8e-45b8-8a94-81ca85f3d7ec -i td/test-erasure-eio/1/new.json 2026-02-12T18:25:38.462 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:25:38.470 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-12T18:25:38.471 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAiG45p/uiRDRAAcgwVlbC1WPTFnAMKFLLsdQ== --osd-uuid e9891a71-9e8e-45b8-8a94-81ca85f3d7ec 2026-02-12T18:25:38.487 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:38.483+0000 7f7ed9655780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:38.488 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:38.484+0000 7f7ed9655780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:38.489 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:38.485+0000 7f7ed9655780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:38.490 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:38.486+0000 7f7ed9655780 -1 bdev(0x398ec00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-12T18:25:38.490 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:38.486+0000 7f7ed9655780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-12T18:25:38.818 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-12T18:25:38.818 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:25:38.819 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-12T18:25:38.819 INFO:tasks.workunit.client.0.trial116.stdout:adding osd1 key to auth repository 2026-02-12T18:25:38.819 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:25:39.154 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-12T18:25:39.154 INFO:tasks.workunit.client.0.trial116.stdout:start osd.1 2026-02-12T18:25:39.155 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:25:39.155 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:25:39.155 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:25:39.155 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:25:39.169 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:39.165+0000 7ff7a564c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:39.170 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:39.167+0000 7ff7a564c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:39.171 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:39.168+0000 7ff7a564c780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:39.307 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:39.303+0000 7ff7a564c780 -1 Falling back to public interface 2026-02-12T18:25:39.382 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-12T18:25:39.382 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:25:39.382 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T18:25:39.382 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:25:39.383 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:25:39.383 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:25:39.383 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:25:39.383 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:25:39.383 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:25:39.384 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:25:39.607 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:39.603+0000 7ff7a564c780 -1 osd.1 0 log_to_monitors true 2026-02-12T18:25:39.610 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:25:40.612 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:25:40.612 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:25:40.613 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:25:40.613 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:25:40.613 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:25:40.613 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:25:40.838 INFO:tasks.workunit.client.0.trial116.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3480133334,v1:127.0.0.1:6811/3480133334] [v2:127.0.0.1:6812/3480133334,v1:127.0.0.1:6813/3480133334] exists,up e9891a71-9e8e-45b8-8a94-81ca85f3d7ec 2026-02-12T18:25:40.841 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:25:40.841 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:25:40.842 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:25:40.842 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:25:40.842 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-12T18:25:40.842 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:25:40.842 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:25:40.842 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-12T18:25:40.842 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:25:40.842 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-12T18:25:40.842 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:25:40.842 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:25:40.842 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:25:40.843 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:25:40.843 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-12T18:25:40.843 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-12T18:25:40.843 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:25:40.843 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:25:40.843 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:25:40.843 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:25:40.843 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:25:40.843 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:25:40.843 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:25:40.843 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:40.843 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:40.844 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:25:40.844 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:25:40.844 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:25:40.844 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:25:40.844 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:25:40.844 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:25:40.844 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:25:40.844 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:25:40.844 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:25:40.844 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:25:40.844 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:25:40.844 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:25:40.844 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:25:40.845 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-12T18:25:40.845 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:25:40.845 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=93f8d427-cf0f-47d4-8aee-75aa895e964e 2026-02-12T18:25:40.845 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 93f8d427-cf0f-47d4-8aee-75aa895e964e' 2026-02-12T18:25:40.846 INFO:tasks.workunit.client.0.trial116.stdout:add osd2 93f8d427-cf0f-47d4-8aee-75aa895e964e 2026-02-12T18:25:40.846 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:25:40.851 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAkG45p7deUMhAADq0BWK8byK5h5EKEI1A/fg== 2026-02-12T18:25:40.852 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAkG45p7deUMhAADq0BWK8byK5h5EKEI1A/fg=="}' 2026-02-12T18:25:40.852 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 93f8d427-cf0f-47d4-8aee-75aa895e964e -i td/test-erasure-eio/2/new.json 2026-02-12T18:25:41.084 INFO:tasks.workunit.client.0.trial116.stdout:2 2026-02-12T18:25:41.092 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-12T18:25:41.093 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAkG45p7deUMhAADq0BWK8byK5h5EKEI1A/fg== --osd-uuid 93f8d427-cf0f-47d4-8aee-75aa895e964e 2026-02-12T18:25:41.109 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:41.105+0000 7fd6f3e3a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:41.111 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:41.107+0000 7fd6f3e3a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:41.112 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:41.108+0000 7fd6f3e3a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:41.112 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:41.108+0000 7fd6f3e3a780 -1 bdev(0x39a8c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-12T18:25:41.112 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:41.108+0000 7fd6f3e3a780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-12T18:25:41.441 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-12T18:25:41.441 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:25:41.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-12T18:25:41.442 INFO:tasks.workunit.client.0.trial116.stdout:adding osd2 key to auth repository 2026-02-12T18:25:41.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:25:41.770 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-12T18:25:41.770 INFO:tasks.workunit.client.0.trial116.stdout:start osd.2 2026-02-12T18:25:41.771 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:25:41.771 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:25:41.771 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:25:41.771 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:25:41.784 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:41.781+0000 7f4db7e54780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:41.786 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:41.782+0000 7f4db7e54780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:41.787 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:41.783+0000 7f4db7e54780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:41.893 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:41.889+0000 7f4db7e54780 -1 Falling back to public interface 2026-02-12T18:25:42.000 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-12T18:25:42.000 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:25:42.001 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T18:25:42.001 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:25:42.001 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:25:42.001 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:25:42.001 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:25:42.001 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:25:42.002 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:25:42.002 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:25:42.235 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:25:42.278 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:42.275+0000 7f4db7e54780 -1 osd.2 0 log_to_monitors true 2026-02-12T18:25:43.236 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:25:43.236 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:25:43.237 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:25:43.237 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:25:43.238 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:25:43.238 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:25:43.464 INFO:tasks.workunit.client.0.trial116.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3608280452,v1:127.0.0.1:6819/3608280452] [v2:127.0.0.1:6820/3608280452,v1:127.0.0.1:6821/3608280452] exists,up 93f8d427-cf0f-47d4-8aee-75aa895e964e 2026-02-12T18:25:43.468 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:25:43.468 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:25:43.468 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:25:43.468 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:25:43.468 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-12T18:25:43.468 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:25:43.468 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:25:43.468 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-12T18:25:43.469 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:25:43.469 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-12T18:25:43.469 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:25:43.469 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:25:43.469 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:25:43.469 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:25:43.469 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-12T18:25:43.469 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-12T18:25:43.469 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:25:43.469 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:25:43.469 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:25:43.469 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:25:43.469 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:25:43.470 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:25:43.470 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:25:43.470 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:43.470 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:43.470 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:25:43.470 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:25:43.470 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:25:43.470 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:25:43.470 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:25:43.470 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:25:43.470 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:25:43.470 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:25:43.470 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:25:43.471 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:25:43.471 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:25:43.471 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:25:43.471 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:25:43.471 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-12T18:25:43.471 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:25:43.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=01cb4bc8-2b34-49dc-a38f-3be5ff86300a 2026-02-12T18:25:43.472 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 01cb4bc8-2b34-49dc-a38f-3be5ff86300a' 2026-02-12T18:25:43.472 INFO:tasks.workunit.client.0.trial116.stdout:add osd3 01cb4bc8-2b34-49dc-a38f-3be5ff86300a 2026-02-12T18:25:43.472 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:25:43.478 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAnG45pbOpNHBAAwh15MEq8VrlalWogsMTb9A== 2026-02-12T18:25:43.478 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAnG45pbOpNHBAAwh15MEq8VrlalWogsMTb9A=="}' 2026-02-12T18:25:43.478 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 01cb4bc8-2b34-49dc-a38f-3be5ff86300a -i td/test-erasure-eio/3/new.json 2026-02-12T18:25:43.714 INFO:tasks.workunit.client.0.trial116.stdout:3 2026-02-12T18:25:43.721 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-12T18:25:43.722 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAnG45pbOpNHBAAwh15MEq8VrlalWogsMTb9A== --osd-uuid 01cb4bc8-2b34-49dc-a38f-3be5ff86300a 2026-02-12T18:25:43.738 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:43.735+0000 7fa185e0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:43.740 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:43.736+0000 7fa185e0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:43.741 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:43.737+0000 7fa185e0f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:43.741 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:43.738+0000 7fa185e0f780 -1 bdev(0x3dbec00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-12T18:25:43.742 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:43.738+0000 7fa185e0f780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-12T18:25:44.088 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-12T18:25:44.088 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:25:44.089 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-12T18:25:44.089 INFO:tasks.workunit.client.0.trial116.stdout:adding osd3 key to auth repository 2026-02-12T18:25:44.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:25:44.408 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-12T18:25:44.409 INFO:tasks.workunit.client.0.trial116.stdout:start osd.3 2026-02-12T18:25:44.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:25:44.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:25:44.409 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:25:44.410 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:25:44.425 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:44.421+0000 7f76b860f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:44.426 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:44.422+0000 7f76b860f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:44.427 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:44.423+0000 7f76b860f780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:44.554 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:44.550+0000 7f76b860f780 -1 Falling back to public interface 2026-02-12T18:25:44.633 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-12T18:25:44.633 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:25:44.633 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-12T18:25:44.633 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:25:44.633 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:25:44.633 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:25:44.633 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:25:44.634 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:25:44.634 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:25:44.634 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:25:44.859 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:44.855+0000 7f76b860f780 -1 osd.3 0 log_to_monitors true 2026-02-12T18:25:44.864 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:25:45.865 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:25:45.866 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:25:45.866 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:25:45.866 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:25:45.867 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:25:45.867 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:25:46.089 INFO:tasks.workunit.client.0.trial116.stdout:osd.3 up in weight 1 up_from 26 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/551198974,v1:127.0.0.1:6827/551198974] [v2:127.0.0.1:6828/551198974,v1:127.0.0.1:6829/551198974] exists,up 01cb4bc8-2b34-49dc-a38f-3be5ff86300a 2026-02-12T18:25:46.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:25:46.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:25:46.090 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:25:46.090 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-12T18:25:46.090 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T18:25:46.090 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T18:25:46.091 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:25:46.091 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:46.091 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:46.091 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.0.asok 2026-02-12T18:25:46.092 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-12T18:25:46.092 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-osd.0.asok log flush 2026-02-12T18:25:46.135 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:25:46.138 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-12T18:25:46.141 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:25:37.011+0000 7fb4f7c29780 0 load: jerasure load: lrc 2026-02-12T18:25:46.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:355: TEST_rados_get_with_subreadall_eio_shard_0: local poolname=pool-jerasure 2026-02-12T18:25:46.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:356: TEST_rados_get_with_subreadall_eio_shard_0: create_erasure_coded_pool pool-jerasure 2 1 2026-02-12T18:25:46.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-12T18:25:46.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-12T18:25:46.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-12T18:25:46.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-12T18:25:46.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-12T18:25:46.142 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-12T18:25:46.143 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-12T18:25:46.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-12T18:25:46.430 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-12T18:25:46.802 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-jerasure' already exists 2026-02-12T18:25:46.809 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:25:47.811 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-12T18:25:47.811 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:25:47.812 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:25:47.812 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:25:47.813 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:25:47.813 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:25:47.813 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:25:47.813 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:25:47.813 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:25:47.813 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:25:47.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:25:47.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:25:47.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:25:47.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:25:47.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:25:47.902 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:25:48.132 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:25:48.132 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:25:48.132 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:25:48.132 INFO:tasks.workunit.client.0.trial116.stderr:3' 2026-02-12T18:25:48.132 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:25:48.132 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:25:48.133 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:25:48.195 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803780 2026-02-12T18:25:48.195 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803780 2026-02-12T18:25:48.196 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780' 2026-02-12T18:25:48.196 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:25:48.196 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:25:48.260 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607556 2026-02-12T18:25:48.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607556 2026-02-12T18:25:48.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-51539607556' 2026-02-12T18:25:48.261 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:25:48.261 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:25:48.325 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411331 2026-02-12T18:25:48.325 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411331 2026-02-12T18:25:48.325 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-51539607556 2-77309411331' 2026-02-12T18:25:48.325 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:25:48.325 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:25:48.389 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149699 2026-02-12T18:25:48.389 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149699 2026-02-12T18:25:48.389 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-51539607556 2-77309411331 3-111669149699' 2026-02-12T18:25:48.389 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:25:48.389 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803780 2026-02-12T18:25:48.389 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:25:48.390 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:25:48.391 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803780 2026-02-12T18:25:48.391 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:25:48.392 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803780 2026-02-12T18:25:48.392 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803780' 2026-02-12T18:25:48.392 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.0 seq 25769803780 2026-02-12T18:25:48.393 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:25:48.617 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803778 -lt 25769803780 2026-02-12T18:25:48.617 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:25:49.618 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:25:49.618 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:25:49.848 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803780 2026-02-12T18:25:49.848 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:25:49.849 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607556 2026-02-12T18:25:49.849 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:25:49.849 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:25:49.850 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607556 2026-02-12T18:25:49.850 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:25:49.851 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607556 2026-02-12T18:25:49.851 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607556' 2026-02-12T18:25:49.852 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.1 seq 51539607556 2026-02-12T18:25:49.852 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:25:50.081 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607556 -lt 51539607556 2026-02-12T18:25:50.081 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:25:50.081 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411331 2026-02-12T18:25:50.082 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:25:50.082 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:25:50.083 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411331 2026-02-12T18:25:50.083 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:25:50.084 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411331 2026-02-12T18:25:50.084 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411331' 2026-02-12T18:25:50.084 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.2 seq 77309411331 2026-02-12T18:25:50.085 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:25:50.313 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411332 -lt 77309411331 2026-02-12T18:25:50.313 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:25:50.314 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149699 2026-02-12T18:25:50.314 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:25:50.315 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:25:50.316 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149699 2026-02-12T18:25:50.316 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:25:50.317 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149699 2026-02-12T18:25:50.317 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149699' 2026-02-12T18:25:50.317 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.3 seq 111669149699 2026-02-12T18:25:50.318 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:25:50.544 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149699 -lt 111669149699 2026-02-12T18:25:50.545 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:25:50.545 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:25:50.546 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:25:50.878 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:25:50.878 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:25:50.878 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:25:50.878 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:25:50.878 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:25:50.878 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:25:50.878 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:25:50.878 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:25:51.103 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:25:51.104 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:25:51.104 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:25:51.104 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:25:51.424 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:25:51.425 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:25:51.425 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:25:51.425 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:358: TEST_rados_get_with_subreadall_eio_shard_0: rados_put_get_data eio td/test-erasure-eio 0 recovery 2026-02-12T18:25:51.425 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-12T18:25:51.425 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-12T18:25:51.425 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-12T18:25:51.425 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-12T18:25:51.425 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=0 2026-02-12T18:25:51.425 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-12T18:25:51.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg=recovery 2026-02-12T18:25:51.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-12T18:25:51.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-76821-0 2026-02-12T18:25:51.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-76821-0 2026-02-12T18:25:51.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-12T18:25:51.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-12T18:25:51.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-76821-0 2026-02-12T18:25:51.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:25:51.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-12T18:25:51.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:25:51.426 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-12T18:25:51.427 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:25:51.427 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-12T18:25:51.427 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:25:51.427 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-12T18:25:51.427 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-76821-0 td/test-erasure-eio/ORIGINAL 2026-02-12T18:25:51.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-76821-0 td/test-erasure-eio 0 2026-02-12T18:25:51.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-12T18:25:51.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-12T18:25:51.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-12T18:25:51.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-12T18:25:51.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-12T18:25:51.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-12T18:25:51.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj-eio-76821-0 2026-02-12T18:25:51.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-12T18:25:51.442 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-12T18:25:51.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-12T18:25:51.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=0 2026-02-12T18:25:51.443 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-12T18:25:51.443 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj-eio-76821-0 2026-02-12T18:25:51.443 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:25:51.443 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-76821-0 2026-02-12T18:25:51.443 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-76821-0 2026-02-12T18:25:51.443 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:25:51.671 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:25:51.671 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:25:51.671 INFO:tasks.workunit.client.0.trial116.stderr:2' 2026-02-12T18:25:51.671 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-12T18:25:51.672 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '2') 2026-02-12T18:25:51.672 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-12T18:25:51.672 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=3 2026-02-12T18:25:51.672 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-12T18:25:51.672 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/3/type 2026-02-12T18:25:51.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-12T18:25:51.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-12T18:25:51.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-12T18:25:51.674 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-12T18:25:51.675 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-12T18:25:51.675 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-12T18:25:51.675 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-12T18:25:51.675 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-12T18:25:51.675 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-12T18:25:51.675 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-12T18:25:51.675 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:25:51.675 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:51.675 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:51.675 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.3.asok 2026-02-12T18:25:51.675 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.76821/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-12T18:25:51.723 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-12T18:25:51.723 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-12T18:25:51.725 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-12T18:25:51.725 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.3 2026-02-12T18:25:51.725 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-12T18:25:51.725 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-12T18:25:51.725 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:25:51.725 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:51.725 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:51.726 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.3.asok 2026-02-12T18:25:51.726 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-12T18:25:51.726 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-osd.3.asok injectdataerr pool-jerasure obj-eio-76821-0 0 2026-02-12T18:25:51.773 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-76821-0 2026-02-12T18:25:51.773 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-12T18:25:51.773 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-12T18:25:51.774 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-76821-0 2026-02-12T18:25:51.774 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-12T18:25:51.774 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-12T18:25:51.774 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-76821-0 td/test-erasure-eio/COPY 2026-02-12T18:25:51.791 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-12T18:25:51.792 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-12T18:25:51.792 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' recovery = recovery ']' 2026-02-12T18:25:51.793 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: get_osds pool-jerasure obj-eio-76821-0 2026-02-12T18:25:51.793 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:25:51.793 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-76821-0 2026-02-12T18:25:51.794 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-76821-0 2026-02-12T18:25:51.794 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:25:52.025 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:25:52.025 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:25:52.025 INFO:tasks.workunit.client.0.trial116.stderr:2' 2026-02-12T18:25:52.025 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-12T18:25:52.025 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: initial_osds=('3' '1' '2') 2026-02-12T18:25:52.026 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: local -a initial_osds 2026-02-12T18:25:52.026 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:171: rados_put_get_data: local last_osd=2 2026-02-12T18:25:52.026 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:173: rados_put_get_data: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-12T18:25:52.026 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:25:52.026 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:25:52.026 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:25:52.026 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:25:52.026 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:25:52.332 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:25:52.332 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:174: rados_put_get_data: ceph osd out 2 2026-02-12T18:25:52.619 INFO:tasks.workunit.client.0.trial116.stderr:osd.2 is already out. 2026-02-12T18:25:52.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: get_osds pool-jerasure obj-eio-76821-0 2026-02-12T18:25:52.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: grep '\<2\>' 2026-02-12T18:25:52.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:25:52.629 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-76821-0 2026-02-12T18:25:52.630 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-76821-0 2026-02-12T18:25:52.630 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:25:52.861 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:25:52.862 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:25:52.862 INFO:tasks.workunit.client.0.trial116.stderr:0' 2026-02-12T18:25:52.862 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 0 2026-02-12T18:25:52.862 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:176: rados_put_get_data: ceph osd in 2 2026-02-12T18:25:53.110 INFO:tasks.workunit.client.0.trial116.stderr:osd.2 is already in. 2026-02-12T18:25:53.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:177: rados_put_get_data: activate_osd td/test-erasure-eio 2 2026-02-12T18:25:53.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-12T18:25:53.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T18:25:53.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-12T18:25:53.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T18:25:53.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-12T18:25:53.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:25:53.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:25:53.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:25:53.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:25:53.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-12T18:25:53.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-12T18:25:53.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T18:25:53.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T18:25:53.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:25:53.122 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T18:25:53.122 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:25:53.122 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:25:53.122 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:25:53.122 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:25:53.122 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:25:53.122 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:25:53.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:25:53.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:25:53.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:25:53.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:25:53.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:25:53.123 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:25:53.123 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:25:53.123 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:25:53.123 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T18:25:53.123 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-12T18:25:53.123 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-12T18:25:53.123 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-12T18:25:53.123 INFO:tasks.workunit.client.0.trial116.stdout:start osd.2 2026-02-12T18:25:53.124 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:25:53.124 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-12T18:25:53.125 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-12T18:25:53.125 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T18:25:53.125 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T18:25:53.125 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T18:25:53.136 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:53.132+0000 7fc17a209780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:53.138 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:53.134+0000 7fc17a209780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:53.139 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:53.135+0000 7fc17a209780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:25:53.288 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:53.285+0000 7fc17a209780 -1 Falling back to public interface 2026-02-12T18:25:53.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-12T18:25:53.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:25:53.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T18:25:53.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:25:53.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:25:53.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:25:53.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:25:53.350 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:25:53.350 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:25:53.350 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:25:53.573 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:25:53.653 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:25:53.649+0000 7fc17a209780 -1 osd.2 33 log_to_monitors true 2026-02-12T18:25:54.574 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:25:54.574 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:25:54.575 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:25:54.575 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:25:54.576 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:25:54.576 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:25:54.801 INFO:tasks.workunit.client.0.trial116.stdout:osd.2 up in weight 1 up_from 42 up_thru 20 down_at 34 last_clean_interval [18,33) [v2:127.0.0.1:6818/2126784422,v1:127.0.0.1:6819/2126784422] [v2:127.0.0.1:6820/2126784422,v1:127.0.0.1:6821/2126784422] exists,up 93f8d427-cf0f-47d4-8aee-75aa895e964e 2026-02-12T18:25:54.803 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:25:54.803 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:25:54.803 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:25:54.803 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:178: rados_put_get_data: wait_for_clean 2026-02-12T18:25:54.803 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:25:54.803 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:25:54.803 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:25:54.804 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:25:54.804 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:25:54.804 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:25:54.804 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:25:54.804 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:25:54.804 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:25:54.889 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:25:54.889 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:25:54.889 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:25:54.889 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:25:54.890 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:25:54.890 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:25:55.116 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:25:55.116 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:25:55.117 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:25:55.117 INFO:tasks.workunit.client.0.trial116.stderr:3' 2026-02-12T18:25:55.117 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:25:55.117 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:25:55.117 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:25:55.184 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803784 2026-02-12T18:25:55.184 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803784 2026-02-12T18:25:55.184 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784' 2026-02-12T18:25:55.184 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:25:55.184 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:25:55.248 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607559 2026-02-12T18:25:55.248 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607559 2026-02-12T18:25:55.248 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-51539607559' 2026-02-12T18:25:55.249 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:25:55.249 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:25:55.313 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626435 2026-02-12T18:25:55.313 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626435 2026-02-12T18:25:55.313 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-51539607559 2-180388626435' 2026-02-12T18:25:55.313 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:25:55.313 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:25:55.378 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149702 2026-02-12T18:25:55.378 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149702 2026-02-12T18:25:55.378 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-51539607559 2-180388626435 3-111669149702' 2026-02-12T18:25:55.378 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:25:55.379 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803784 2026-02-12T18:25:55.379 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:25:55.379 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:25:55.380 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803784 2026-02-12T18:25:55.380 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:25:55.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803784 2026-02-12T18:25:55.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-12T18:25:55.381 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.0 seq 25769803784 2026-02-12T18:25:55.382 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:25:55.606 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803784 2026-02-12T18:25:55.606 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:25:56.607 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:25:56.608 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:25:56.834 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803784 2026-02-12T18:25:56.834 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:25:57.835 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-12T18:25:57.835 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:25:58.062 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-12T18:25:58.063 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:25:58.063 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607559 2026-02-12T18:25:58.063 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:25:58.064 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:25:58.065 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607559 2026-02-12T18:25:58.065 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:25:58.066 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607559 2026-02-12T18:25:58.066 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607559' 2026-02-12T18:25:58.066 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.1 seq 51539607559 2026-02-12T18:25:58.066 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:25:58.293 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607560 -lt 51539607559 2026-02-12T18:25:58.293 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:25:58.294 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-180388626435 2026-02-12T18:25:58.294 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:25:58.294 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:25:58.295 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-180388626435 2026-02-12T18:25:58.295 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:25:58.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=180388626435 2026-02-12T18:25:58.296 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 180388626435' 2026-02-12T18:25:58.297 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.2 seq 180388626435 2026-02-12T18:25:58.297 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:25:58.529 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 180388626435 -lt 180388626435 2026-02-12T18:25:58.529 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:25:58.530 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149702 2026-02-12T18:25:58.530 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:25:58.530 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:25:58.531 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149702 2026-02-12T18:25:58.531 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:25:58.533 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149702 2026-02-12T18:25:58.533 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149702' 2026-02-12T18:25:58.533 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.3 seq 111669149702 2026-02-12T18:25:58.533 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:25:58.758 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149703 -lt 111669149702 2026-02-12T18:25:58.758 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:25:58.759 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:25:58.759 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:25:59.076 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:25:59.076 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:25:59.077 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:25:59.078 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:25:59.078 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:25:59.078 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:25:59.078 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:25:59.078 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:25:59.300 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:25:59.301 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:25:59.301 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:25:59.301 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:25:59.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:25:59.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:25:59.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:25:59.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:360: TEST_rados_get_with_subreadall_eio_shard_0: delete_erasure_coded_pool pool-jerasure 2026-02-12T18:25:59.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-12T18:25:59.621 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-12T18:25:59.906 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-jerasure' does not exist 2026-02-12T18:25:59.914 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-12T18:26:00.211 INFO:tasks.workunit.client.0.trial116.stderr:erasure-code-profile myprofile does not exist 2026-02-12T18:26:00.220 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-12T18:26:00.220 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-12T18:26:00.220 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:26:00.220 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-12T18:26:00.221 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:26:00.221 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:26:00.221 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:26:00.221 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:26:00.221 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:26:00.329 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:26:00.330 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:26:00.330 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:26:00.331 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:26:00.332 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:26:00.333 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:26:00.333 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:26:00.334 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:26:00.334 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:26:00.334 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:26:00.334 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:26:00.335 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:26:00.336 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:26:00.336 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-12T18:26:00.346 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:26:00.347 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:26:00.347 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:26:00.347 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76821 2026-02-12T18:26:00.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:26:00.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:26:00.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-12T18:26:00.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-12T18:26:00.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-12T18:26:00.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-12T18:26:00.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-12T18:26:00.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:26:00.349 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-12T18:26:00.349 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:26:00.350 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:26:00.350 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:26:00.350 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:26:00.350 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:26:00.350 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:26:00.351 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:26:00.351 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:26:00.352 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:26:00.353 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:26:00.353 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:26:00.353 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:26:00.354 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:26:00.354 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:26:00.354 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:26:00.355 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:26:00.355 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:26:00.356 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:26:00.356 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-12T18:26:00.357 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:26:00.357 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:26:00.357 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:26:00.358 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76821 2026-02-12T18:26:00.358 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:26:00.358 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:26:00.358 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-12T18:26:00.360 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T18:26:00.360 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:26:00.360 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:26:00.360 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76821 2026-02-12T18:26:00.361 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T18:26:00.362 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T18:26:00.362 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T18:26:00.362 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-12T18:26:00.362 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-12T18:26:00.362 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-12T18:26:00.362 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T18:26:00.363 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T18:26:00.363 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T18:26:00.363 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-12T18:26:00.363 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-12T18:26:00.380 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T18:26:00.380 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:26:00.380 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:26:00.380 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:26:00.380 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:26:00.380 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:26:00.381 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:26:00.381 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T18:26:00.405 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T18:26:00.406 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T18:26:00.406 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:26:00.406 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:26:00.406 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T18:26:00.407 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T18:26:00.407 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:26:00.407 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:26:00.407 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:26:00.408 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:26:00.408 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:26:00.408 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:26:00.408 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:26:00.408 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:26:00.408 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76821/ceph-mon.a.asok config get fsid 2026-02-12T18:26:00.449 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T18:26:00.450 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T18:26:00.450 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T18:26:00.450 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T18:26:00.450 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T18:26:00.451 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T18:26:00.451 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:26:00.451 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:26:00.452 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:26:00.452 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:26:00.452 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:26:00.452 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:26:00.452 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T18:26:00.452 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76821/ceph-mon.a.asok config get mon_host 2026-02-12T18:26:00.494 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-12T18:26:00.494 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-12T18:26:00.494 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T18:26:00.494 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T18:26:00.494 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T18:26:00.495 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-12T18:26:00.495 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T18:26:00.594 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T18:26:00.594 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:26:00.594 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:26:00.594 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:26:00.594 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:26:00.595 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:26:00.595 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:26:00.595 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:26:00.596 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T18:26:00.611 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-12T18:26:00.611 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T18:26:00.706 INFO:tasks.workunit.client.0.trial116.stderr:pool 'rbd' already exists 2026-02-12T18:26:00.713 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:26:01.715 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-12T18:26:01.716 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T18:26:01.716 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T18:26:01.716 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:26:01.716 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:26:01.716 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:26:01.716 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-mon.a.asok 2026-02-12T18:26:01.716 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-12T18:26:01.716 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-mon.a.asok log flush 2026-02-12T18:26:01.755 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:26:01.760 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-12T18:26:01.761 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:26:00.394+0000 7f7a31048c00 0 load: jerasure load: lrc 2026-02-12T18:26:01.762 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_with_subreadall_eio_shard_1 td/test-erasure-eio 2026-02-12T18:26:01.762 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:364: TEST_rados_get_with_subreadall_eio_shard_1: local dir=td/test-erasure-eio 2026-02-12T18:26:01.762 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:365: TEST_rados_get_with_subreadall_eio_shard_1: local shard_id=1 2026-02-12T18:26:01.762 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:367: TEST_rados_get_with_subreadall_eio_shard_1: setup_osds 4 2026-02-12T18:26:01.762 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-12T18:26:01.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-12T18:26:01.763 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-12T18:26:01.763 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-12T18:26:01.767 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:26:01.767 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-12T18:26:01.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:26:01.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:26:01.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T18:26:01.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:26:01.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-12T18:26:01.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:26:01.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:26:01.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:26:01.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:26:01.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-12T18:26:01.768 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-12T18:26:01.769 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:26:01.769 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:26:01.769 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:26:01.769 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:26:01.769 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:26:01.769 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:26:01.769 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:26:01.769 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:26:01.769 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:26:01.769 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:26:01.769 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:26:01.769 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:26:01.769 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:26:01.770 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:26:01.770 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:26:01.770 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:26:01.770 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:26:01.770 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:26:01.770 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:26:01.770 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:26:01.770 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:26:01.770 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:26:01.770 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-12T18:26:01.770 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:26:01.771 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=abd2dedf-06b7-4445-9bb4-094694861450 2026-02-12T18:26:01.771 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 abd2dedf-06b7-4445-9bb4-094694861450' 2026-02-12T18:26:01.771 INFO:tasks.workunit.client.0.trial116.stdout:add osd0 abd2dedf-06b7-4445-9bb4-094694861450 2026-02-12T18:26:01.772 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:26:01.778 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA5G45p6NEoLhAAQLsgGlNVoTy/C38Dsp9GHw== 2026-02-12T18:26:01.778 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA5G45p6NEoLhAAQLsgGlNVoTy/C38Dsp9GHw=="}' 2026-02-12T18:26:01.778 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new abd2dedf-06b7-4445-9bb4-094694861450 -i td/test-erasure-eio/0/new.json 2026-02-12T18:26:01.880 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:26:01.886 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-12T18:26:01.887 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA5G45p6NEoLhAAQLsgGlNVoTy/C38Dsp9GHw== --osd-uuid abd2dedf-06b7-4445-9bb4-094694861450 2026-02-12T18:26:01.903 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:01.899+0000 7ff8ee82d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:26:01.904 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:01.901+0000 7ff8ee82d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:26:01.905 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:01.901+0000 7ff8ee82d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:26:01.905 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:01.901+0000 7ff8ee82d780 -1 bdev(0x24ecc00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-12T18:26:01.906 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:01.902+0000 7ff8ee82d780 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-12T18:26:02.375 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-12T18:26:02.375 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:26:02.376 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T18:26:02.376 INFO:tasks.workunit.client.0.trial116.stdout:adding osd0 key to auth repository 2026-02-12T18:26:02.376 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:26:02.477 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T18:26:02.477 INFO:tasks.workunit.client.0.trial116.stdout:start osd.0 2026-02-12T18:26:02.478 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:26:02.478 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:26:02.478 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:26:02.478 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:26:02.492 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:02.488+0000 7ff40ec50780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:26:02.493 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:02.490+0000 7ff40ec50780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:26:02.494 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:02.491+0000 7ff40ec50780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:26:02.573 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T18:26:02.574 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:26:02.574 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T18:26:02.574 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:26:02.574 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:26:02.574 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:26:02.574 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:26:02.574 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:26:02.575 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:26:02.575 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:26:02.668 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:26:02.688 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:02.685+0000 7ff40ec50780 -1 Falling back to public interface 2026-02-12T18:26:02.941 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:02.937+0000 7ff40ec50780 -1 osd.0 0 log_to_monitors true 2026-02-12T18:26:03.669 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:26:03.670 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:26:03.670 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:26:03.670 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:26:03.671 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:26:03.671 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T18:26:03.897 INFO:tasks.workunit.client.0.trial116.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/619139095,v1:127.0.0.1:6803/619139095] [v2:127.0.0.1:6804/619139095,v1:127.0.0.1:6805/619139095] exists,up abd2dedf-06b7-4445-9bb4-094694861450 2026-02-12T18:26:03.901 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:26:03.901 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:26:03.901 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:26:03.901 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:26:03.901 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-12T18:26:03.901 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:26:03.901 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:26:03.901 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-12T18:26:03.901 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:26:03.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-12T18:26:03.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:26:03.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:26:03.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:26:03.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:26:03.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-12T18:26:03.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-12T18:26:03.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:26:03.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:26:03.902 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:26:03.902 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:26:03.902 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:26:03.902 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:26:03.903 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:26:03.903 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:26:03.903 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:26:03.903 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:26:03.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:26:03.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:26:03.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:26:03.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:26:03.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:26:03.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:26:03.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:26:03.903 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:26:03.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:26:03.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:26:03.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:26:03.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:26:03.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-12T18:26:03.904 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:26:03.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d114dd57-e8b1-4f81-a8b8-33026dbdaeea 2026-02-12T18:26:03.904 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 d114dd57-e8b1-4f81-a8b8-33026dbdaeea' 2026-02-12T18:26:03.904 INFO:tasks.workunit.client.0.trial116.stdout:add osd1 d114dd57-e8b1-4f81-a8b8-33026dbdaeea 2026-02-12T18:26:03.905 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:26:03.911 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA7G45pfiYcNhAAwyEbR4KKEQ1DzeHxsTLRLg== 2026-02-12T18:26:03.911 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA7G45pfiYcNhAAwyEbR4KKEQ1DzeHxsTLRLg=="}' 2026-02-12T18:26:03.911 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d114dd57-e8b1-4f81-a8b8-33026dbdaeea -i td/test-erasure-eio/1/new.json 2026-02-12T18:26:04.145 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:26:04.152 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-12T18:26:04.153 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA7G45pfiYcNhAAwyEbR4KKEQ1DzeHxsTLRLg== --osd-uuid d114dd57-e8b1-4f81-a8b8-33026dbdaeea 2026-02-12T18:26:04.170 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:04.166+0000 7f52c4c3b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:26:04.171 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:04.167+0000 7f52c4c3b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:26:04.172 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:04.168+0000 7f52c4c3b780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:26:04.173 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:04.169+0000 7f52c4c3b780 -1 bdev(0x2624c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-12T18:26:04.173 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:04.169+0000 7f52c4c3b780 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-12T18:26:04.497 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-12T18:26:04.497 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:26:04.497 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-12T18:26:04.498 INFO:tasks.workunit.client.0.trial116.stdout:adding osd1 key to auth repository 2026-02-12T18:26:04.498 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:26:04.822 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-12T18:26:04.823 INFO:tasks.workunit.client.0.trial116.stdout:start osd.1 2026-02-12T18:26:04.824 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:26:04.824 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:26:04.824 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:26:04.824 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:26:04.837 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:04.833+0000 7f00fda3a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:26:04.838 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:04.834+0000 7f00fda3a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:26:04.839 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:04.835+0000 7f00fda3a780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:26:04.965 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:04.962+0000 7f00fda3a780 -1 Falling back to public interface 2026-02-12T18:26:05.050 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-12T18:26:05.050 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:26:05.050 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T18:26:05.051 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:26:05.051 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:26:05.051 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:26:05.051 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:26:05.051 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:26:05.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:26:05.052 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:26:05.275 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:05.271+0000 7f00fda3a780 -1 osd.1 0 log_to_monitors true 2026-02-12T18:26:05.277 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:26:06.279 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:26:06.279 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:26:06.279 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:26:06.280 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:26:06.280 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:26:06.280 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T18:26:06.507 INFO:tasks.workunit.client.0.trial116.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1891497879,v1:127.0.0.1:6811/1891497879] [v2:127.0.0.1:6812/1891497879,v1:127.0.0.1:6813/1891497879] exists,up d114dd57-e8b1-4f81-a8b8-33026dbdaeea 2026-02-12T18:26:06.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:26:06.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:26:06.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:26:06.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:26:06.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-12T18:26:06.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:26:06.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:26:06.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-12T18:26:06.511 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:26:06.512 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-12T18:26:06.512 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:26:06.512 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:26:06.512 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:26:06.512 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:26:06.512 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-12T18:26:06.512 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-12T18:26:06.512 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:26:06.512 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:26:06.512 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:26:06.512 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:26:06.512 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:26:06.512 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:26:06.513 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:26:06.513 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:26:06.513 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:26:06.513 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:26:06.513 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:26:06.513 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:26:06.513 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:26:06.513 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:26:06.513 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:26:06.513 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:26:06.513 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:26:06.513 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:26:06.513 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:26:06.514 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:26:06.514 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:26:06.514 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:26:06.514 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-12T18:26:06.514 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:26:06.514 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f3f7f078-fb03-42db-9ded-ac70b32f1b23 2026-02-12T18:26:06.514 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 f3f7f078-fb03-42db-9ded-ac70b32f1b23' 2026-02-12T18:26:06.514 INFO:tasks.workunit.client.0.trial116.stdout:add osd2 f3f7f078-fb03-42db-9ded-ac70b32f1b23 2026-02-12T18:26:06.515 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:26:06.520 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA+G45psOfWHhAAkqDuy3fggsGkrNX8U1eYEA== 2026-02-12T18:26:06.520 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA+G45psOfWHhAAkqDuy3fggsGkrNX8U1eYEA=="}' 2026-02-12T18:26:06.521 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f3f7f078-fb03-42db-9ded-ac70b32f1b23 -i td/test-erasure-eio/2/new.json 2026-02-12T18:26:06.751 INFO:tasks.workunit.client.0.trial116.stdout:2 2026-02-12T18:26:06.759 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-12T18:26:06.760 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA+G45psOfWHhAAkqDuy3fggsGkrNX8U1eYEA== --osd-uuid f3f7f078-fb03-42db-9ded-ac70b32f1b23 2026-02-12T18:26:06.776 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:06.772+0000 7f429b3c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:26:06.777 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:06.774+0000 7f429b3c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:26:06.778 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:06.774+0000 7f429b3c6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:26:06.779 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:06.775+0000 7f429b3c6780 -1 bdev(0x3dc8c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-12T18:26:06.779 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:06.775+0000 7f429b3c6780 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-12T18:26:07.099 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-12T18:26:07.099 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:26:07.100 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-12T18:26:07.100 INFO:tasks.workunit.client.0.trial116.stdout:adding osd2 key to auth repository 2026-02-12T18:26:07.100 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:26:07.438 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-12T18:26:07.438 INFO:tasks.workunit.client.0.trial116.stdout:start osd.2 2026-02-12T18:26:07.439 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:26:07.439 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:26:07.439 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:26:07.440 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:26:07.453 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:07.449+0000 7f76d4c33780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:26:07.454 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:07.450+0000 7f76d4c33780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:26:07.455 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:07.451+0000 7f76d4c33780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:26:07.642 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:07.638+0000 7f76d4c33780 -1 Falling back to public interface 2026-02-12T18:26:07.666 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-12T18:26:07.667 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:26:07.667 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T18:26:07.667 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:26:07.667 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:26:07.667 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:26:07.667 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:26:07.667 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:26:07.668 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:26:07.668 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:26:07.892 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:26:07.981 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:07.979+0000 7f76d4c33780 -1 osd.2 0 log_to_monitors true 2026-02-12T18:26:08.893 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:26:08.894 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:26:08.894 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:26:08.895 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:26:08.895 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:26:08.895 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:26:09.119 INFO:tasks.workunit.client.0.trial116.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3744266752,v1:127.0.0.1:6819/3744266752] [v2:127.0.0.1:6820/3744266752,v1:127.0.0.1:6821/3744266752] exists,up f3f7f078-fb03-42db-9ded-ac70b32f1b23 2026-02-12T18:26:09.123 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:26:09.123 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:26:09.123 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:26:09.123 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-12T18:26:09.124 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-12T18:26:09.124 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-12T18:26:09.124 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T18:26:09.124 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-12T18:26:09.124 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T18:26:09.124 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-12T18:26:09.124 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:26:09.124 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:26:09.124 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:26:09.124 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:26:09.124 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-12T18:26:09.124 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-12T18:26:09.125 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T18:26:09.125 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T18:26:09.125 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:26:09.125 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T18:26:09.125 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:26:09.125 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:26:09.125 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:26:09.125 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:26:09.125 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:26:09.125 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:26:09.125 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:26:09.125 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:26:09.126 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T18:26:09.126 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T18:26:09.126 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:26:09.126 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:26:09.126 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:26:09.126 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:26:09.126 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:26:09.126 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:26:09.126 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T18:26:09.126 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T18:26:09.126 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-12T18:26:09.126 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T18:26:09.126 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0b661c8d-840c-4f36-bd9c-d0fbe2232293 2026-02-12T18:26:09.127 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 0b661c8d-840c-4f36-bd9c-d0fbe2232293' 2026-02-12T18:26:09.127 INFO:tasks.workunit.client.0.trial116.stdout:add osd3 0b661c8d-840c-4f36-bd9c-d0fbe2232293 2026-02-12T18:26:09.127 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T18:26:09.134 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBBG45phCjMBxAAfnpvcu0SeruB05qWYb/YMA== 2026-02-12T18:26:09.134 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBBG45phCjMBxAAfnpvcu0SeruB05qWYb/YMA=="}' 2026-02-12T18:26:09.134 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0b661c8d-840c-4f36-bd9c-d0fbe2232293 -i td/test-erasure-eio/3/new.json 2026-02-12T18:26:09.364 INFO:tasks.workunit.client.0.trial116.stdout:3 2026-02-12T18:26:09.371 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-12T18:26:09.372 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBBG45phCjMBxAAfnpvcu0SeruB05qWYb/YMA== --osd-uuid 0b661c8d-840c-4f36-bd9c-d0fbe2232293 2026-02-12T18:26:09.388 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:09.385+0000 7f087045d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:26:09.389 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:09.386+0000 7f087045d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:26:09.390 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:09.387+0000 7f087045d780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:26:09.390 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:09.388+0000 7f087045d780 -1 bdev(0x3560c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-12T18:26:09.391 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:09.388+0000 7f087045d780 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-12T18:26:09.713 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-12T18:26:09.713 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T18:26:09.714 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-12T18:26:09.714 INFO:tasks.workunit.client.0.trial116.stdout:adding osd3 key to auth repository 2026-02-12T18:26:09.714 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T18:26:10.030 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-12T18:26:10.030 INFO:tasks.workunit.client.0.trial116.stdout:start osd.3 2026-02-12T18:26:10.031 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:26:10.031 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T18:26:10.031 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T18:26:10.031 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T18:26:10.045 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:10.042+0000 7f4231bd6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:26:10.046 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:10.043+0000 7f4231bd6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:26:10.047 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:10.044+0000 7f4231bd6780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:26:10.199 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:10.196+0000 7f4231bd6780 -1 Falling back to public interface 2026-02-12T18:26:10.256 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-12T18:26:10.256 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:26:10.256 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-12T18:26:10.257 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:26:10.257 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:26:10.257 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:26:10.257 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:26:10.257 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:26:10.258 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:26:10.258 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:26:10.484 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:26:10.546 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:10.543+0000 7f4231bd6780 -1 osd.3 0 log_to_monitors true 2026-02-12T18:26:11.485 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:26:11.485 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:26:11.486 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:26:11.486 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:26:11.487 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:26:11.487 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T18:26:11.710 INFO:tasks.workunit.client.0.trial116.stdout:osd.3 up in weight 1 up_from 26 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2542551741,v1:127.0.0.1:6827/2542551741] [v2:127.0.0.1:6828/2542551741,v1:127.0.0.1:6829/2542551741] exists,up 0b661c8d-840c-4f36-bd9c-d0fbe2232293 2026-02-12T18:26:11.711 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:26:11.711 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:26:11.711 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:26:11.711 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-12T18:26:11.712 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T18:26:11.712 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T18:26:11.712 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:26:11.712 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:26:11.712 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:26:11.712 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.0.asok 2026-02-12T18:26:11.712 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-12T18:26:11.712 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-osd.0.asok log flush 2026-02-12T18:26:11.756 INFO:tasks.workunit.client.0.trial116.stdout:{} 2026-02-12T18:26:11.760 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-12T18:26:11.762 INFO:tasks.workunit.client.0.trial116.stdout:2026-02-12T18:26:02.687+0000 7ff40ec50780 0 load: jerasure load: lrc 2026-02-12T18:26:11.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:369: TEST_rados_get_with_subreadall_eio_shard_1: local poolname=pool-jerasure 2026-02-12T18:26:11.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:370: TEST_rados_get_with_subreadall_eio_shard_1: create_erasure_coded_pool pool-jerasure 2 1 2026-02-12T18:26:11.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-12T18:26:11.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-12T18:26:11.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-12T18:26:11.763 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-12T18:26:11.764 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-12T18:26:11.764 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-12T18:26:11.764 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-12T18:26:12.048 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-12T18:26:12.048 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-12T18:26:12.423 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-jerasure' already exists 2026-02-12T18:26:12.432 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T18:26:13.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-12T18:26:13.434 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:26:13.434 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:26:13.434 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:26:13.435 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:26:13.435 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:26:13.435 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:26:13.435 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:26:13.435 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:26:13.435 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:26:13.523 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:26:13.523 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:26:13.523 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:26:13.523 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:26:13.523 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:26:13.523 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:26:13.750 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:26:13.750 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:26:13.750 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:26:13.750 INFO:tasks.workunit.client.0.trial116.stderr:3' 2026-02-12T18:26:13.750 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:26:13.750 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:26:13.750 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:26:13.814 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803780 2026-02-12T18:26:13.814 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803780 2026-02-12T18:26:13.814 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780' 2026-02-12T18:26:13.815 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:26:13.815 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:26:13.877 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607556 2026-02-12T18:26:13.877 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607556 2026-02-12T18:26:13.877 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-51539607556' 2026-02-12T18:26:13.877 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:26:13.878 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:26:13.941 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411331 2026-02-12T18:26:13.941 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411331 2026-02-12T18:26:13.941 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-51539607556 2-77309411331' 2026-02-12T18:26:13.941 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:26:13.941 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:26:14.005 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149699 2026-02-12T18:26:14.006 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149699 2026-02-12T18:26:14.006 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803780 1-51539607556 2-77309411331 3-111669149699' 2026-02-12T18:26:14.006 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:26:14.006 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803780 2026-02-12T18:26:14.006 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:26:14.007 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:26:14.007 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803780 2026-02-12T18:26:14.008 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:26:14.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803780 2026-02-12T18:26:14.009 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803780' 2026-02-12T18:26:14.009 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.0 seq 25769803780 2026-02-12T18:26:14.009 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:26:14.238 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803778 -lt 25769803780 2026-02-12T18:26:14.238 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:26:15.239 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:26:15.240 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:26:15.466 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803781 -lt 25769803780 2026-02-12T18:26:15.467 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:26:15.467 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607556 2026-02-12T18:26:15.467 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:26:15.468 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:26:15.469 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607556 2026-02-12T18:26:15.469 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:26:15.470 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607556 2026-02-12T18:26:15.470 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607556' 2026-02-12T18:26:15.470 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.1 seq 51539607556 2026-02-12T18:26:15.471 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:26:15.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607556 -lt 51539607556 2026-02-12T18:26:15.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:26:15.707 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-77309411331 2026-02-12T18:26:15.707 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:26:15.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:26:15.708 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411331 2026-02-12T18:26:15.708 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:26:15.709 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411331 2026-02-12T18:26:15.709 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 77309411331' 2026-02-12T18:26:15.710 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.2 seq 77309411331 2026-02-12T18:26:15.710 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:26:15.934 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411332 -lt 77309411331 2026-02-12T18:26:15.935 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:26:15.935 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149699 2026-02-12T18:26:15.935 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:26:15.936 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:26:15.936 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149699 2026-02-12T18:26:15.937 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:26:15.938 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149699 2026-02-12T18:26:15.938 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149699' 2026-02-12T18:26:15.938 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.3 seq 111669149699 2026-02-12T18:26:15.939 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:26:16.162 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149699 -lt 111669149699 2026-02-12T18:26:16.163 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:26:16.163 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:26:16.164 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:26:16.480 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:26:16.480 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:26:16.481 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:26:16.481 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:26:16.481 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:26:16.481 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:26:16.481 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:26:16.481 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:26:16.705 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:26:16.706 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:26:16.706 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:26:16.706 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:26:17.026 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:26:17.026 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:26:17.026 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:26:17.026 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:372: TEST_rados_get_with_subreadall_eio_shard_1: rados_put_get_data eio td/test-erasure-eio 1 recovery 2026-02-12T18:26:17.026 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-12T18:26:17.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-12T18:26:17.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-12T18:26:17.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-12T18:26:17.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=1 2026-02-12T18:26:17.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-12T18:26:17.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg=recovery 2026-02-12T18:26:17.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-12T18:26:17.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-76821-1 2026-02-12T18:26:17.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-76821-1 2026-02-12T18:26:17.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-12T18:26:17.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-12T18:26:17.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-76821-1 2026-02-12T18:26:17.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:26:17.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-12T18:26:17.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:26:17.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-12T18:26:17.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:26:17.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-12T18:26:17.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-12T18:26:17.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-12T18:26:17.027 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-76821-1 td/test-erasure-eio/ORIGINAL 2026-02-12T18:26:17.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-76821-1 td/test-erasure-eio 1 2026-02-12T18:26:17.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2480: inject_eio: local pooltype=ec 2026-02-12T18:26:17.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: shift 2026-02-12T18:26:17.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: local which=data 2026-02-12T18:26:17.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: shift 2026-02-12T18:26:17.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: local poolname=pool-jerasure 2026-02-12T18:26:17.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: shift 2026-02-12T18:26:17.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: local objname=obj-eio-76821-1 2026-02-12T18:26:17.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: shift 2026-02-12T18:26:17.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: local dir=td/test-erasure-eio 2026-02-12T18:26:17.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: shift 2026-02-12T18:26:17.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: local shard_id=1 2026-02-12T18:26:17.044 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: shift 2026-02-12T18:26:17.044 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: get_osds pool-jerasure obj-eio-76821-1 2026-02-12T18:26:17.044 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:26:17.044 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-76821-1 2026-02-12T18:26:17.045 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-76821-1 2026-02-12T18:26:17.045 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:26:17.272 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:26:17.272 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:26:17.272 INFO:tasks.workunit.client.0.trial116.stderr:2' 2026-02-12T18:26:17.272 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-12T18:26:17.273 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: initial_osds=('3' '1' '2') 2026-02-12T18:26:17.273 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2493: inject_eio: local -a initial_osds 2026-02-12T18:26:17.273 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local osd_id=1 2026-02-12T18:26:17.273 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: '[' ec '!=' ec ']' 2026-02-12T18:26:17.273 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: cat td/test-erasure-eio/1/type 2026-02-12T18:26:17.274 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: inject_eio: type=bluestore 2026-02-12T18:26:17.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-12T18:26:17.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-12T18:26:17.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-12T18:26:17.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-12T18:26:17.275 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-12T18:26:17.275 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-12T18:26:17.275 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-12T18:26:17.275 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-12T18:26:17.275 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-12T18:26:17.275 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:26:17.275 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:26:17.276 INFO:tasks.workunit.client.0.trial116.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:26:17.276 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.1.asok 2026-02-12T18:26:17.276 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.76821/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-12T18:26:17.322 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-12T18:26:17.322 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: local loop=0 2026-02-12T18:26:17.323 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: grep -q Invalid 2026-02-12T18:26:17.324 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: get_asok_path osd.1 2026-02-12T18:26:17.324 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-12T18:26:17.324 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-12T18:26:17.324 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T18:26:17.324 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:26:17.324 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:26:17.324 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76821/ceph-osd.1.asok 2026-02-12T18:26:17.324 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: CEPH_ARGS= 2026-02-12T18:26:17.324 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: ceph --admin-daemon /tmp/ceph-asok.76821/ceph-osd.1.asok injectdataerr pool-jerasure obj-eio-76821-1 1 2026-02-12T18:26:17.372 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-76821-1 2026-02-12T18:26:17.373 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-12T18:26:17.373 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-12T18:26:17.373 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-76821-1 2026-02-12T18:26:17.373 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-12T18:26:17.373 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-12T18:26:17.373 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-76821-1 td/test-erasure-eio/COPY 2026-02-12T18:26:17.389 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-12T18:26:17.390 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-12T18:26:17.390 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' recovery = recovery ']' 2026-02-12T18:26:17.391 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: get_osds pool-jerasure obj-eio-76821-1 2026-02-12T18:26:17.392 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:26:17.392 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-76821-1 2026-02-12T18:26:17.392 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-76821-1 2026-02-12T18:26:17.393 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:26:17.618 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:26:17.618 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:26:17.618 INFO:tasks.workunit.client.0.trial116.stderr:2' 2026-02-12T18:26:17.619 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-12T18:26:17.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: initial_osds=('3' '1' '2') 2026-02-12T18:26:17.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: local -a initial_osds 2026-02-12T18:26:17.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:171: rados_put_get_data: local last_osd=2 2026-02-12T18:26:17.619 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:173: rados_put_get_data: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-12T18:26:17.619 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:26:17.619 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:26:17.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:26:17.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:26:17.620 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:26:17.925 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:26:17.926 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:174: rados_put_get_data: ceph osd out 2 2026-02-12T18:26:18.206 INFO:tasks.workunit.client.0.trial116.stderr:osd.2 is already out. 2026-02-12T18:26:18.214 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: get_osds pool-jerasure obj-eio-76821-1 2026-02-12T18:26:18.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: grep '\<2\>' 2026-02-12T18:26:18.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-12T18:26:18.215 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-76821-1 2026-02-12T18:26:18.215 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-76821-1 2026-02-12T18:26:18.215 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T18:26:18.447 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-12T18:26:18.447 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:26:18.447 INFO:tasks.workunit.client.0.trial116.stderr:0' 2026-02-12T18:26:18.447 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 0 2026-02-12T18:26:18.448 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:176: rados_put_get_data: ceph osd in 2 2026-02-12T18:26:18.692 INFO:tasks.workunit.client.0.trial116.stderr:osd.2 is already in. 2026-02-12T18:26:18.704 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:177: rados_put_get_data: activate_osd td/test-erasure-eio 2 2026-02-12T18:26:18.704 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-12T18:26:18.704 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T18:26:18.704 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-12T18:26:18.704 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T18:26:18.704 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-12T18:26:18.704 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-12T18:26:18.704 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T18:26:18.704 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T18:26:18.705 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T18:26:18.705 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-12T18:26:18.705 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-12T18:26:18.705 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T18:26:18.705 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T18:26:18.705 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-12T18:26:18.705 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T18:26:18.705 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T18:26:18.705 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T18:26:18.705 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T18:26:18.705 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:26:18.705 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:26:18.706 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:26:18.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' 2026-02-12T18:26:18.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T18:26:18.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-12T18:26:18.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-12T18:26:18.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T18:26:18.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T18:26:18.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T18:26:18.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T18:26:18.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T18:26:18.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-12T18:26:18.706 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-12T18:26:18.707 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-12T18:26:18.707 INFO:tasks.workunit.client.0.trial116.stdout:start osd.2 2026-02-12T18:26:18.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=d0590416-029f-4eec-9478-b4dec50a4b77 --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.76821/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T18:26:18.708 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-12T18:26:18.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-12T18:26:18.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T18:26:18.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T18:26:18.708 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T18:26:18.717 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:18.714+0000 7f1772c23780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:26:18.719 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:18.715+0000 7f1772c23780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:26:18.720 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:18.716+0000 7f1772c23780 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T18:26:18.861 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:18.858+0000 7f1772c23780 -1 Falling back to public interface 2026-02-12T18:26:18.937 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-12T18:26:18.937 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T18:26:18.937 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T18:26:18.937 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T18:26:18.937 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T18:26:18.937 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:26:18.938 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T18:26:18.938 INFO:tasks.workunit.client.0.trial116.stdout:0 2026-02-12T18:26:18.938 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:26:18.938 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:26:19.169 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T18:26:19.208 INFO:tasks.workunit.client.0.trial116.stderr:2026-02-12T18:26:19.205+0000 7f1772c23780 -1 osd.2 34 log_to_monitors true 2026-02-12T18:26:20.170 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T18:26:20.171 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T18:26:20.171 INFO:tasks.workunit.client.0.trial116.stdout:1 2026-02-12T18:26:20.172 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T18:26:20.172 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T18:26:20.172 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T18:26:20.401 INFO:tasks.workunit.client.0.trial116.stdout:osd.2 up in weight 1 up_from 43 up_thru 20 down_at 35 last_clean_interval [18,34) [v2:127.0.0.1:6818/2506656004,v1:127.0.0.1:6819/2506656004] [v2:127.0.0.1:6820/2506656004,v1:127.0.0.1:6821/2506656004] exists,up f3f7f078-fb03-42db-9ded-ac70b32f1b23 2026-02-12T18:26:20.403 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T18:26:20.403 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T18:26:20.403 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T18:26:20.403 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:178: rados_put_get_data: wait_for_clean 2026-02-12T18:26:20.403 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T18:26:20.403 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T18:26:20.403 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T18:26:20.403 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T18:26:20.404 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T18:26:20.404 INFO:tasks.workunit.client.0.trial116.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T18:26:20.404 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T18:26:20.404 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T18:26:20.404 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T18:26:20.490 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T18:26:20.490 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T18:26:20.490 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T18:26:20.490 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T18:26:20.490 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T18:26:20.490 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T18:26:20.713 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T18:26:20.713 INFO:tasks.workunit.client.0.trial116.stderr:1 2026-02-12T18:26:20.713 INFO:tasks.workunit.client.0.trial116.stderr:2 2026-02-12T18:26:20.713 INFO:tasks.workunit.client.0.trial116.stderr:3' 2026-02-12T18:26:20.713 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T18:26:20.713 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:26:20.714 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T18:26:20.776 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803784 2026-02-12T18:26:20.776 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803784 2026-02-12T18:26:20.776 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784' 2026-02-12T18:26:20.776 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:26:20.777 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T18:26:20.844 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607559 2026-02-12T18:26:20.844 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607559 2026-02-12T18:26:20.844 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-51539607559' 2026-02-12T18:26:20.844 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:26:20.844 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T18:26:20.907 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593731 2026-02-12T18:26:20.907 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593731 2026-02-12T18:26:20.907 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-51539607559 2-184683593731' 2026-02-12T18:26:20.907 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T18:26:20.907 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T18:26:20.973 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149702 2026-02-12T18:26:20.973 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149702 2026-02-12T18:26:20.973 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-25769803784 1-51539607559 2-184683593731 3-111669149702' 2026-02-12T18:26:20.973 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:26:20.974 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-25769803784 2026-02-12T18:26:20.974 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:26:20.974 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T18:26:20.975 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803784 2026-02-12T18:26:20.975 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:26:20.976 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=25769803784 2026-02-12T18:26:20.976 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-12T18:26:20.977 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.0 seq 25769803784 2026-02-12T18:26:20.977 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:26:21.204 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803784 2026-02-12T18:26:21.205 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:26:22.205 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T18:26:22.206 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:26:22.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803782 -lt 25769803784 2026-02-12T18:26:22.433 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T18:26:23.434 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-12T18:26:23.435 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T18:26:23.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-12T18:26:23.664 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:26:23.664 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-51539607559 2026-02-12T18:26:23.664 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:26:23.665 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T18:26:23.666 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607559 2026-02-12T18:26:23.666 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:26:23.667 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=51539607559 2026-02-12T18:26:23.668 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 51539607559' 2026-02-12T18:26:23.668 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.1 seq 51539607559 2026-02-12T18:26:23.668 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T18:26:23.892 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 51539607560 -lt 51539607559 2026-02-12T18:26:23.892 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:26:23.893 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-184683593731 2026-02-12T18:26:23.893 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:26:23.894 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T18:26:23.895 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-184683593731 2026-02-12T18:26:23.895 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:26:23.896 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=184683593731 2026-02-12T18:26:23.896 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.2 seq 184683593731 2026-02-12T18:26:23.897 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 184683593731' 2026-02-12T18:26:23.897 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T18:26:24.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 184683593731 -lt 184683593731 2026-02-12T18:26:24.122 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T18:26:24.123 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-111669149702 2026-02-12T18:26:24.123 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T18:26:24.124 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T18:26:24.124 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149702 2026-02-12T18:26:24.125 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T18:26:24.126 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=111669149702 2026-02-12T18:26:24.126 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 111669149702' 2026-02-12T18:26:24.126 INFO:tasks.workunit.client.0.trial116.stdout:waiting osd.3 seq 111669149702 2026-02-12T18:26:24.126 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T18:26:24.348 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 111669149703 -lt 111669149702 2026-02-12T18:26:24.349 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T18:26:24.349 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:26:24.349 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:26:24.668 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-12T18:26:24.668 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T18:26:24.669 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T18:26:24.669 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T18:26:24.669 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T18:26:24.670 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T18:26:24.670 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T18:26:24.670 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T18:26:24.887 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-12T18:26:24.888 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T18:26:24.888 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T18:26:24.888 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T18:26:25.207 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-12T18:26:25.207 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T18:26:25.207 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T18:26:25.207 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:374: TEST_rados_get_with_subreadall_eio_shard_1: delete_erasure_coded_pool pool-jerasure 2026-02-12T18:26:25.207 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-12T18:26:25.207 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-12T18:26:25.478 INFO:tasks.workunit.client.0.trial116.stderr:pool 'pool-jerasure' does not exist 2026-02-12T18:26:25.486 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-12T18:26:25.787 INFO:tasks.workunit.client.0.trial116.stderr:erasure-code-profile myprofile does not exist 2026-02-12T18:26:25.796 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-12T18:26:25.796 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-12T18:26:25.796 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T18:26:25.796 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-12T18:26:25.797 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:26:25.797 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:26:25.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:26:25.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:26:25.797 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:26:25.905 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:26:25.906 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:26:25.906 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:26:25.907 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:26:25.908 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:26:25.909 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:26:25.909 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:26:25.910 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:26:25.910 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:26:25.910 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:26:25.910 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:26:25.911 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:26:25.912 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T18:26:25.913 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-12T18:26:25.923 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:26:25.923 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:26:25.923 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:26:25.923 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76821 2026-02-12T18:26:25.925 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:26:25.925 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:26:25.925 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-12T18:26:25.925 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/test-erasure-eio 0 2026-02-12T18:26:25.925 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-12T18:26:25.925 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-12T18:26:25.925 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-12T18:26:25.925 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T18:26:25.925 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T18:26:25.925 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T18:26:25.925 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T18:26:25.926 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T18:26:25.926 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T18:26:25.927 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T18:26:25.927 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T18:26:25.928 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T18:26:25.929 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T18:26:25.929 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T18:26:25.929 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T18:26:25.930 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:26:25.930 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T18:26:25.931 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T18:26:25.931 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T18:26:25.931 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:26:25.932 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-12T18:26:25.933 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-12T18:26:25.934 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T18:26:25.934 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T18:26:25.934 INFO:tasks.workunit.client.0.trial116.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76821 2026-02-12T18:26:25.934 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76821 2026-02-12T18:26:25.936 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T18:26:25.936 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T18:26:25.936 INFO:tasks.workunit.client.0.trial116.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-12T18:26:25.937 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-12T18:26:25.937 DEBUG:teuthology.orchestra.run.trial116:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-12T18:26:25.965 INFO:tasks.workunit:Stopping ['erasure-code'] on client.0... 2026-02-12T18:26:25.965 DEBUG:teuthology.orchestra.run.trial116:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-12T18:26:26.327 DEBUG:teuthology.parallel:result is None 2026-02-12T18:26:26.327 DEBUG:teuthology.orchestra.run.trial116:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-12T18:26:26.352 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-12T18:26:26.352 DEBUG:teuthology.orchestra.run.trial116:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-12T18:26:26.409 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-12T18:26:26.409 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-12T18:26:26.424 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-12T18:26:26.424 DEBUG:teuthology.orchestra.run.trial116:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-12T18:26:26.579 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-12T18:26:26.592 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-12T18:26:26.592 DEBUG:teuthology.orchestra.run.trial116:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-12T18:26:26.608 INFO:teuthology.orchestra.run.trial116.stderr:bash: line 1: ntpq: command not found 2026-02-12T18:26:26.628 INFO:teuthology.orchestra.run.trial116.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-12T18:26:26.628 INFO:teuthology.orchestra.run.trial116.stdout:=============================================================================== 2026-02-12T18:26:26.629 INFO:teuthology.orchestra.run.trial116.stdout:^- gopher.fart.website 3 6 377 30 -2985us[-2985us] +/- 94ms 2026-02-12T18:26:26.629 INFO:teuthology.orchestra.run.trial116.stdout:^* time.ecansol.net 1 6 377 93 +1485us[+1203us] +/- 15ms 2026-02-12T18:26:26.629 INFO:teuthology.orchestra.run.trial116.stdout:^+ sensei.ruselabs.com 2 6 377 31 -2906us[-2906us] +/- 37ms 2026-02-12T18:26:26.629 INFO:teuthology.orchestra.run.trial116.stdout:^- 172-232-15-202.ip.linode> 2 6 377 29 -9780us[-9780us] +/- 78ms 2026-02-12T18:26:26.629 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-12T18:26:26.641 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-12T18:26:26.641 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-12T18:26:26.653 DEBUG:teuthology.orchestra.run.trial116:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-12T18:26:26.712 DEBUG:teuthology.orchestra.run.trial116:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-12T18:26:26.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-12T18:26:26.777 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-12T18:26:26.791 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-12T18:26:26.805 INFO:teuthology.task.internal:Duration was 947.795242 seconds 2026-02-12T18:26:26.806 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-12T18:26:26.820 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-12T18:26:26.820 DEBUG:teuthology.orchestra.run.trial116:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-12T18:26:26.858 INFO:teuthology.orchestra.run.trial116.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-12T18:26:27.007 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-12T18:26:27.008 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial116.front.sepia.ceph.com 2026-02-12T18:26:27.008 DEBUG:teuthology.orchestra.run.trial116:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-12T18:26:27.066 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-12T18:26:27.067 DEBUG:teuthology.orchestra.run.trial116:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-12T18:26:27.148 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-12T18:26:27.148 DEBUG:teuthology.orchestra.run.trial116:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-12T18:26:27.211 INFO:teuthology.orchestra.run.trial116.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-12T18:26:27.212 INFO:teuthology.orchestra.run.trial116.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-12T18:26:27.212 INFO:teuthology.orchestra.run.trial116.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-12T18:26:27.212 INFO:teuthology.orchestra.run.trial116.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-12T18:26:27.212 INFO:teuthology.orchestra.run.trial116.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-12T18:26:27.215 INFO:teuthology.orchestra.run.trial116.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 86.9% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-12T18:26:27.217 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-12T18:26:27.231 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-12T18:26:27.232 DEBUG:teuthology.orchestra.run.trial116:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-12T18:26:27.282 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-12T18:26:27.300 DEBUG:teuthology.orchestra.run.trial116:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:26:27.344 INFO:teuthology.orchestra.run.trial116.stdout:kernel.core_pattern = core 2026-02-12T18:26:27.357 DEBUG:teuthology.orchestra.run.trial116:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-12T18:26:27.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-12T18:26:27.412 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-12T18:26:27.428 INFO:teuthology.task.internal:Transferring archived files... 2026-02-12T18:26:27.430 DEBUG:teuthology.misc:Transferring archived files from trial116:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/yuriw-2026-02-12_15:38:14-rados-wip-yuri4-testing-2026-02-11-2126-distro-default-trial/46524/remote/trial116 2026-02-12T18:26:27.432 DEBUG:teuthology.orchestra.run.trial116:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-12T18:26:27.486 INFO:teuthology.task.internal:Removing archive directory... 2026-02-12T18:26:27.487 DEBUG:teuthology.orchestra.run.trial116:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-12T18:26:27.535 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-12T18:26:27.549 INFO:teuthology.task.internal:Not uploading archives. 2026-02-12T18:26:27.549 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-12T18:26:27.564 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-12T18:26:27.565 DEBUG:teuthology.orchestra.run.trial116:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-12T18:26:27.592 INFO:teuthology.orchestra.run.trial116.stdout: 786525 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 12 18:26 /home/ubuntu/cephtest 2026-02-12T18:26:27.593 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-12T18:26:27.609 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-12T18:26:27.645 INFO:teuthology.run:Summary data: description: rados/standalone/{supported-random-distro$/{centos_latest} workloads/erasure-code} duration: 947.7952418327332 flavor: default owner: scheduled_yuriw@soko04.front.sepia.ceph.com success: true 2026-02-12T18:26:27.645 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-12T18:26:27.762 INFO:teuthology.run:pass